What if a SQL query didn't run on one database, but on dozens of small serverless functions at once? A distributed query engine on Azure, and an honest answer to whether that's faster.
The idea
Big spatial and temporal datasets, like years of bike rides across a city, make for slow queries. The pitch: instead of one database chewing through everything, cut the query into a grid of small pieces by area and by time, run every piece in its own serverless function at the same time, and stitch the results back together.
FaaSTLane was a five-person project for the Distributed Systems Prototyping course at TU Berlin. A small CLI fires off a single query or a whole benchmark, and you choose how many workers to throw at it.
My part: the coordinator
I worked mostly on the piece that decides how a query gets cut up:
- Parsing the SQL and building an execution plan from it,
HAVINGclauses included. - Digging the spatial and temporal ranges out of the
WHEREclause (BETWEEN,ST_Distance,ST_Within) so they can be sliced into tasks. - Loading dataset metadata from Blob storage and checking the query against it.
- The project’s first infrastructure-as-code setup.
Spoiler: it wasn’t faster
Splitting the query and merging the results ate up everything the parallel pieces saved. Not the headline we hoped for, but still a result: spreading work across serverless functions has a price, and now we know where it gets paid.