education

FaaSTLane

Distributed Systems Prototyping, TU Berlin

What if a SQL query didn't run on one database, but on dozens of small serverless functions at once? A distributed query engine on Azure, and an honest answer to whether that's faster.

The idea

Big spatial and temporal datasets, like years of bike rides across a city, make for slow queries. The pitch: instead of one database chewing through everything, cut the query into a grid of small pieces by area and by time, run every piece in its own serverless function at the same time, and stitch the results back together.

FaaSTLane was a five-person project for the Distributed Systems Prototyping course at TU Berlin. A small CLI fires off a single query or a whole benchmark, and you choose how many workers to throw at it.

My part: the coordinator

I worked mostly on the piece that decides how a query gets cut up:

  • Parsing the SQL and building an execution plan from it, HAVING clauses included.
  • Digging the spatial and temporal ranges out of the WHERE clause (BETWEEN, ST_Distance, ST_Within) so they can be sliced into tasks.
  • Loading dataset metadata from Blob storage and checking the query against it.
  • The project’s first infrastructure-as-code setup.

Spoiler: it wasn’t faster

Splitting the query and merging the results ate up everything the parallel pieces saved. Not the headline we hoped for, but still a result: spreading work across serverless functions has a price, and now we know where it gets paid.

How it’s built

CLIENTPLANDurable FunctionsFAN OUTSTORAGEBlob storagefaastlane.shexec · benchmarkStarterHTTP triggerCoordinatororchestratorSQL processorWHERE → ranges,HAVINGTask executorsone per grid cellWorkersDuckDB + spatialMetadatadataset boundsDatasetsSimRa bike ridesResultsone file per task1query2bounds3s × t tasks4read slicewrite5download
  1. 1The CLI sends a SQL query plus how many spatial and temporal workers to use.
  2. 2The coordinator parses the SQL and reads the time and space ranges out of the WHERE clause, checked against the dataset's metadata.
  3. 3It cuts those ranges into a grid of spatial × temporal tasks and starts one executor per cell.
  4. 4Each worker runs its slice of the query in DuckDB, straight on the data in Blob storage.
  5. 5The partial results are written back and can be downloaded with the CLI.
Built on Azure Durable Functions. We ran the same benchmark queries against a single PostGIS database to compare.