🔧 Fix APIs & tile services Performance & scale Serving Spatial Data · step 18 of 20 Updated

Fixing a Spatial API That Is Slow Under Load

Problem statement

The endpoint responds in 40 ms when tested by hand and times out when ten people use it. That gap is not mysterious once the numbers are measured, and for spatial APIs the cause is almost always one of four things.

Measured on a single-worker FastAPI service with ten concurrent clients:

endpoint                        payload      throughput   p50 latency
50-feature page                 857 kB          32.5 rps     296.7 ms
1,000-feature page             12.43 MB          2.3 rps   4,360.5 ms
blocking call in async def      216 kB          15.7 rps     628.0 ms
same work, async                216 kB          66.7 rps     144.9 ms
precomputed body + ETag         340 kB         100.2 rps      94.6 ms
same, answering 304                  0 B      1,191.0 rps       6.2 ms

Four separate fixes, each worth between four and twelve times, and none of them requires a bigger machine.

Quick answer

Work down this list; each step is cheaper than the one after it:

# 1. is a blocking call inside an async endpoint?   4.2× in measurement
async def handler(): time.sleep(0.05)          # blocks the event loop
async def handler(): await anyio.sleep(0.05)   # does not

# 2. is the response too large?                     14× between page sizes
limit: int = Query(50, ge=1, le=200)           # not le=10000

# 3. is anything cached?                           11.9× on the 304 path
headers = {"ETag": etag, "Cache-Control": "public, max-age=300"}

# 4. only then, add workers
uvicorn service:app --workers 8

Adding workers first is the common instinct and the least effective: eight workers each serialising a 12.4 MB response are eight times as slow as one worker serialising 857 kB.

Bar chart of throughput for four spatial API performance problems and their fixes.
Eight workers each serialising 12.4 MB are eight times the problem.

Step-by-step solution

1. Find the blocking call in the async path

An async def endpoint runs on the event loop. Any synchronous call inside it — a database driver, requests, time.sleep, a heavy to_json() — blocks every other request in that worker for its duration.

Measured with a 50 ms synchronous call inside async def: 15.7 requests per second at 628 ms. The same work with an awaitable sleep: 66.7 requests per second at 145 ms. A factor of 4.2, from one keyword.

Two fixes:

# use an async library
async with httpx.AsyncClient() as client:
    response = await client.get(url)

# or move the sync work off the loop
from starlette.concurrency import run_in_threadpool
result = await run_in_threadpool(expensive_sync_function, argument)

A plain def endpoint in FastAPI is already run in a thread pool, so the simplest fix for a synchronous handler is to remove the async.

2. Measure the payload, because it is usually the answer

The measured throughput difference between a 857 kB page and a 12.4 MB page was 32.5 against 2.3 requests per second — fourteen times, on the same server, for the same endpoint.

That is not a server problem. Serialising and transmitting fourteen times as much data takes fourteen times as long. Reduce it:

  • Cap the page size. 50 features, not 1,000.
  • Round the coordinates. Six decimal places took a measured payload from 54.06 MB to 29.56 MB — a 45% cut.
  • Send only the properties the client uses.
  • Enable gzip. 857,061 bytes became 299,123 on the wire, a factor of 2.87.

3. Cache, because the cheapest request is one that does no work

Measured: the same precomputed body served 100.2 requests per second at 94.6 ms, and the conditional request answered with 304 served 1,191 requests per second at 6.2 ms.

Twelve times, for an ETag header. And a response inside its max-age window costs the server nothing at all, because the client does not send a request.

Derive the ETag from a data version plus the query parameters, so the 304 path never touches the data.

4. Profile the handler before optimising the query

The instinct is to blame the database. Time the parts:

import time
from contextlib import contextmanager


@contextmanager
def timed(label, into):
    start = time.perf_counter()
    yield
    into[label] = time.perf_counter() - start


timings = {}
with timed("query", timings):
    subset = layer.cx[x0:x1, y0:y1]
with timed("serialise", timings):
    body = subset.to_json()
print(timings)

On a GeoPandas-backed service the serialisation is frequently larger than the query: measured, a bounding-box selection over 4,596 polygons took 0.43 ms, while producing 857 kB of GeoJSON took hundreds of times that.

5. Then scale out, with the right worker count 

uvicorn service:app --workers $(( $(nproc) )) --loop uvloop --http httptools

Workers multiply throughput for CPU-bound serialisation and multiply memory. A service holding a 500 MB GeoDataFrame in module scope needs 500 MB per worker unless the data is shared or moved to a database.

Check memory before raising the worker count, or the fix is an out-of-memory kill.

6. Put a cache in front

A CDN or a caching proxy converts repeated identical requests into zero application requests. For tiles at versioned URLs marked immutable, that is close to all of them.

This is the step that changes the shape of the load rather than the speed of the handler, and it is why the caching headers in step 3 matter more than any code optimisation.

Two panels contrasting a blocked event loop with concurrent request handling.
A 50 ms synchronous call stops every other request in that worker for 50 ms.

Code examples

Example 1 — finding blocking calls in async endpoints 

import asyncio
import time
import warnings


def install_blocking_detector(threshold_ms=100):
    """Warn when the event loop is blocked longer than the threshold."""
    loop = asyncio.get_event_loop()
    loop.set_debug(True)
    loop.slow_callback_duration = threshold_ms / 1000

    async def monitor():
        while True:
            started = time.perf_counter()
            await asyncio.sleep(0.1)
            drift = (time.perf_counter() - started - 0.1) * 1000
            if drift > threshold_ms:
                warnings.warn(f"event loop blocked for {drift:.0f} ms — "
                              f"a synchronous call in an async endpoint")
    asyncio.create_task(monitor())

The drift measurement is the reliable signal: a loop that should have slept 100 ms and slept 700 ms was blocked for 600 ms by something synchronous.

Example 2 — a load test that reports the numbers that matter 

import asyncio
import statistics
import time
import httpx


async def load_test(url, n=200, concurrency=10, headers=None):
    limits = httpx.Limits(max_connections=concurrency)
    async with httpx.AsyncClient(limits=limits, timeout=60) as client:
        await client.get(url, headers=headers or {})          # warm

        semaphore = asyncio.Semaphore(concurrency)
        latencies = []

        async def one():
            async with semaphore:
                started = time.perf_counter()
                response = await client.get(url, headers=headers or {})
                latencies.append(time.perf_counter() - started)
                return response

        started = time.perf_counter()
        responses = await asyncio.gather(*[one() for _ in range(n)])
        wall = time.perf_counter() - started

    latencies.sort()
    size = len(responses[0].content)
    print(f"{url}")
    print(f"  {n} requests, {concurrency} concurrent")
    print(f"  {n / wall:8.1f} rps   p50 {statistics.median(latencies) * 1000:7.1f} ms"
          f"   p95 {latencies[int(0.95 * len(latencies)) - 1] * 1000:7.1f} ms")
    print(f"  {size:,} bytes per response, "
          f"{size * n / wall / 1e6:.1f} MB/s sustained")

Reporting the sustained megabytes per second is what makes the payload problem obvious: a service pushing 28 MB/s is not slow, it is saturated.

Example 3 — the fixes, applied 

import hashlib
import os
from fastapi import FastAPI, Query, Request, Response
from fastapi.middleware.gzip import GZipMiddleware
import geopandas as gpd
import shapely

app = FastAPI()
app.add_middleware(GZipMiddleware, minimum_size=1000)          # fix 2

LAYER = gpd.read_file("provinces.gpkg")
LAYER.sindex                                                    # build once
LAYER_VERSION = os.environ.get("LAYER_VERSION", "1")


@app.get("/features")
def features(request: Request, bbox: str | None = None,
             limit: int = Query(50, ge=1, le=200)):             # fix 2: a real cap
    seed = f"{LAYER_VERSION}|{bbox}|{limit}"
    etag = '"' + hashlib.sha256(seed.encode()).hexdigest()[:16] + '"'
    if request.headers.get("if-none-match") == etag:            # fix 3
        return Response(status_code=304, headers={"ETag": etag})

    subset = LAYER
    if bbox:
        x0, y0, x1, y1 = (float(v) for v in bbox.split(","))
        subset = subset.cx[x0:x1, y0:y1]

    page = subset.iloc[:limit].copy()
    page["geometry"] = shapely.set_precision(page.geometry.values, 1e-6)  # fix 2
    return Response(page.to_json(), media_type="application/geo+json",
                    headers={"ETag": etag,
                             "Cache-Control": "public, max-age=300",
                             "Vary": "Accept-Encoding"})

Note that the endpoint is def, not async def. FastAPI runs synchronous handlers in a thread pool, so GeoPandas work does not block the event loop — which is the correct default for a library that has no async interface.

Explanation

Why a blocking call is so much worse than it looks

A single worker's event loop is one thread. A 50 ms synchronous call does not slow that request by 50 ms — it stops every other request in that worker for 50 ms.

With ten concurrent clients the effect compounds into the measured 15.7 requests per second, against 66.7 for the same work done without blocking. The fix is either an async library, run_in_threadpool, or removing the async keyword so FastAPI handles the threading.

Why payload size dominates spatial APIs

Most REST APIs return a few kilobytes. A spatial API returns geometry, and the measured spread is enormous: a 50-feature page at 857 kB, a 1,000-feature page at 12.4 MB, an unfiltered dataset at 54 MB.

Serialisation is CPU work proportional to the output, and transfer is bandwidth proportional to the output. Both scale linearly with the payload, which is why the throughput ratio (32.5 to 2.3) tracks the size ratio (857 kB to 12.4 MB) so closely.

That is also why a bigger machine helps less than expected: it does not make the response smaller.

Why caching beats every code optimisation

The measured 304 path served 1,191 requests per second because it did essentially nothing: compare a string, return a header. No query, no serialisation, no body.

No amount of query tuning approaches that, because the fastest possible query is still slower than not running one. And within a max-age window the request does not even arrive.

Caching is therefore the first thing to add and the last thing to remove, and its effectiveness is why response-size and blocking fixes should be judged on the uncached path only.

Why more workers is the last step

Workers multiply throughput and memory. A service holding a 500 MB layer in module scope costs 500 MB per worker; eight workers is 4 GB before any request is served.

They also do not fix the two largest problems. Eight workers each blocked on a synchronous call are eight blocked workers, and eight workers each serialising 12.4 MB saturate the network rather than the CPU. Fix the handler, then scale it.

Five ordered steps for fixing a slow spatial API, ending with adding workers.
A bounding-box selection took 0.43 ms; producing 857 kB of GeoJSON took far longer.

Edge cases or notes

  • A def endpoint in FastAPI runs in a thread pool; async def does not. Choose deliberately.
  • run_in_threadpool is the escape hatch for synchronous work inside an async handler.
  • Module-level GeoDataFrames are per worker. Check memory before scaling out.
  • Build sindex at start-up, not on the first request.
  • Measure sustained MB/s, not just requests per second — it makes saturation obvious.
  • Warm the service before load testing, or the first request's imports dominate.
  • --loop uvloop --http httptools is a small free improvement in uvicorn.
  • Judge fixes on the uncached path, then add the cache back.

FAQ

Why is my API fast in testing and slow under load?

Usually one of four things: a blocking call in an async endpoint, an oversized response, no caching, or a single worker. Measured, those were worth 4.2×, 14×, 11.9× and the worker count respectively.

How much does a blocking call cost?

Measured with a 50 ms synchronous call inside async def: 15.7 requests per second against 66.7 for the same work done asynchronously.

Should I add more workers?

Last. Eight workers each blocked, or each serialising 12.4 MB, are eight times the problem. Fix the handler first, and check memory — module-level data is per worker.

What is the single biggest improvement?

Caching. A conditional request answered with 304 ran at 1,191 requests per second against 100 for generating the same body.

How do I know whether the payload is the problem?

Compare throughput at two page sizes. Measured, 50 features gave 32.5 requests per second and 1,000 gave 2.3 — the ratio tracks the payload almost exactly.

Is async def always better?

No. FastAPI runs plain def handlers in a thread pool, which is the right choice for synchronous libraries such as GeoPandas. async def is only better when the work is genuinely awaitable.