Fixing a Spatial API That Is Slow Under Load
On this page
Problem statement
The endpoint responds in 40 ms when tested by hand and times out when ten people use it. That gap is not mysterious once the numbers are measured, and for spatial APIs the cause is almost always one of four things.
Measured on a single-worker FastAPI service serving Natural Earth's 4,596 provinces to ten concurrent clients (Example 4):
endpoint payload throughput p50 latency
50-feature page, serialised 1.01 MB 23.1 rps 422.1 ms
1,000-feature page, serialised 15.59 MB 2.6 rps 3,465.5 ms
50 ms blocking call, async def 1.01 MB 19.8 rps 504.4 ms
the same 50 ms, awaited 1.01 MB 175.6 rps 54.1 ms
precomputed body + ETag 1.01 MB 586.4 rps 16.3 ms
same, answering 304 0 B 1,318.3 rps 5.3 ms
Four separate fixes, worth between 2.2 and 25 times each, and none of them requires a bigger machine.
Quick answer
Work down this list; each step is cheaper than the one after it:
# 1. is a blocking call inside an async endpoint? 8.9× in measurement
async def handler(): time.sleep(0.05) # blocks the event loop
async def handler(): await anyio.sleep(0.05) # does not
# 2. is the response too large? 8.9× between page sizes
limit: int = Query(50, ge=1, le=200) # not le=10000
# 3. is anything cached? 25× for a stored body, 2.2× more for a 304
headers = {"ETag": etag, "Cache-Control": "public, max-age=300"}
# 4. only then, add workers
uvicorn service:app --workers 8Adding workers first is the common instinct and the least effective. One worker served 2.6 requests a second of the 1,000-feature page and 23.1 of the 50-feature one; eight workers on the large page would, at perfectly linear scaling, still manage fewer than one worker on the small page.
Blocking calls, payload size, caching, then workers
1. Find the blocking call in the async path
An async def endpoint runs on the event loop. Any synchronous call inside it — a database driver, requests, time.sleep, a heavy to_json() — blocks every other request in that worker for its duration.
Measured with a 50 ms synchronous call inside async def: 19.8 requests per second at 504 ms — one request every 50 ms, however many clients are waiting. The same 50 ms awaited: 175.6 requests per second at 54 ms. A factor of 8.9, from one call.
Two fixes:
# use an async library
async with httpx.AsyncClient() as client:
response = await client.get(url)
# or move the sync work off the loop
from starlette.concurrency import run_in_threadpool
result = await run_in_threadpool(expensive_sync_function, argument)A plain def endpoint in FastAPI is already run in a thread pool, so the simplest fix for a synchronous handler is to remove the async.
2. Measure the payload, because it is usually the answer
The measured throughput difference between a 1.01 MB page and a 15.59 MB page was 23.1 against 2.6 requests per second — 8.9 times, on the same server, for the same endpoint.
That is not a server problem. Serialising and transmitting 15 times as much data took nearly nine times as long per request. Reduce it:
- Cap the page size. 50 features, not 1,000.
- Round the coordinates. Six decimal places took the whole layer from 68.35 MB of GeoJSON to 46.44 MB — a 32% cut.
- Send only the properties the client uses.
- Enable gzip. The 50-feature page's 1,010,943 bytes became 337,521, a factor of 3.00.
3. Cache, because the cheapest request is one that does no work
Measured: serialising the 50-feature page on every request served 23.1 requests per second; storing the body once served 586.4 per second at 16.3 ms, and the conditional request answered with 304 served 1,318.3 per second at 5.3 ms.
Twenty-five times for computing the body once, and another 2.2 times for an ETag header. And a response inside its max-age window costs the server nothing at all, because the client does not send a request.
Derive the ETag from a data version plus the query parameters, so the 304 path never touches the data.
4. Profile the handler before optimising the query
The instinct is to blame the database. Time the parts:
import time
from contextlib import contextmanager
@contextmanager
def timed(label, into):
start = time.perf_counter()
yield
into[label] = time.perf_counter() - start
x0, y0, x1, y1 = -10, 35, 30, 60 # a bounding box over Europe
layer.sindex # built once, at start-up
timings = {}
with timed("query", timings):
subset = layer.cx[x0:x1, y0:y1]
with timed("serialise", timings):
body = subset.to_json()
print(f"{len(subset):,} features; query {timings['query'] * 1000:.2f} ms, "
f"serialise {timings['serialise'] * 1000:,.0f} ms, {len(body) / 1e6:.2f} MB")1,613 features; query 4.39 ms, serialise 351 ms, 15.56 MB
On a GeoPandas-backed service the serialisation is frequently larger than the query: measured, a bounding-box selection over Europe from 4,596 polygons took 4.39 ms, while serialising the 1,613 features it found, 15.56 MB of GeoJSON, took 351 ms — 80 times as long.
5. Then scale out, with the right worker count
uvicorn service:app --workers $(( $(nproc) )) --loop uvloop --http httptoolsWorkers multiply throughput for CPU-bound serialisation and multiply memory. A service holding a 500 MB GeoDataFrame in module scope needs 500 MB per worker unless the data is shared or moved to a database.
Check memory before raising the worker count, or the fix is an out-of-memory kill.
6. Put a cache in front
A CDN or a caching proxy converts repeated identical requests into zero application requests. For tiles at versioned URLs marked immutable, that is close to all of them.
This is the step that changes the shape of the load rather than the speed of the handler, and it is why the caching headers in step 3 matter more than any code optimisation.
Finding blockers and load-testing the fixes
The data used on this page
provinces.gpkg is Natural Earth's 1:10m admin-1 states and provinces layer: 4,596 polygons with 121 attribute columns. Example 4 serves it from a single uvicorn worker in a separate process and loads it with Example 2's test, ten concurrent clients on the same machine, with the load average printed beside the timings.
import os
import geopandas as gpd
if not os.path.exists("provinces.gpkg"):
gpd.read_file("https://naciscdn.org/naturalearth/10m/cultural/ne_10m_admin_1_states_provinces.zip") \
.to_file("provinces.gpkg")
layer = gpd.read_file("provinces.gpkg")
print(len(layer), "polygons,", len(layer.columns) - 1, "attribute columns")4596 polygons, 121 attribute columns
Example 1 — finding blocking calls in async endpoints
import asyncio
import time
import warnings
def install_blocking_detector(threshold_ms=100):
"""Warn when the event loop is blocked longer than the threshold."""
loop = asyncio.get_event_loop()
loop.set_debug(True)
loop.slow_callback_duration = threshold_ms / 1000
async def monitor():
while True:
started = time.perf_counter()
await asyncio.sleep(0.1)
drift = (time.perf_counter() - started - 0.1) * 1000
if drift > threshold_ms:
warnings.warn(f"event loop blocked for {drift:.0f} ms — "
f"a synchronous call in an async endpoint")
asyncio.create_task(monitor())
async def demo():
install_blocking_detector(threshold_ms=100)
await asyncio.sleep(0.15) # let the monitor start
time.sleep(0.7) # a synchronous call on the loop
await asyncio.sleep(0.3)
with warnings.catch_warnings(record=True) as caught:
warnings.simplefilter("always")
asyncio.run(demo())
print([str(w.message) for w in caught if "blocked" in str(w.message)])['event loop blocked for 651 ms — a synchronous call in an async endpoint']
The drift measurement is the reliable signal: a loop that should have slept 100 ms and slept 700 ms was blocked for 600 ms by something synchronous.
Example 2 — a load test that reports the numbers that matter
import asyncio
import statistics
import time
import httpx
async def load_test(url, n=200, concurrency=10, headers=None):
limits = httpx.Limits(max_connections=concurrency)
async with httpx.AsyncClient(limits=limits, timeout=60) as client:
await client.get(url, headers=headers or {}) # warm
semaphore = asyncio.Semaphore(concurrency)
latencies = []
async def one():
async with semaphore:
started = time.perf_counter()
response = await client.get(url, headers=headers or {})
latencies.append(time.perf_counter() - started)
return response
started = time.perf_counter()
responses = await asyncio.gather(*[one() for _ in range(n)])
wall = time.perf_counter() - started
latencies.sort()
size = len(responses[0].content)
print(f"{url}")
print(f" {n} requests, {concurrency} concurrent")
print(f" {n / wall:8.1f} rps p50 {statistics.median(latencies) * 1000:7.1f} ms"
f" p95 {latencies[int(0.95 * len(latencies)) - 1] * 1000:7.1f} ms")
print(f" {size:,} bytes per response, "
f"{size * n / wall / 1e6:.1f} MB/s sustained")Reporting the sustained megabytes per second is what makes the payload problem obvious: the 1,000-feature page sustained 40.1 MB/s at 2.6 requests per second. That service is not slow; it is busy producing bytes.
Example 3 — the fixes, applied
import hashlib
import os
from fastapi import FastAPI, Query, Request, Response
from fastapi.middleware.gzip import GZipMiddleware
import geopandas as gpd
import shapely
app = FastAPI()
app.add_middleware(GZipMiddleware, minimum_size=1000) # fix 2
LAYER = gpd.read_file("provinces.gpkg")
LAYER.sindex # build once
LAYER_VERSION = os.environ.get("LAYER_VERSION", "1")
@app.get("/features")
def features(request: Request, bbox: str | None = None,
limit: int = Query(50, ge=1, le=200)): # fix 2: a real cap
seed = f"{LAYER_VERSION}|{bbox}|{limit}"
etag = '"' + hashlib.sha256(seed.encode()).hexdigest()[:16] + '"'
if request.headers.get("if-none-match") == etag: # fix 3
return Response(status_code=304, headers={"ETag": etag})
subset = LAYER
if bbox:
x0, y0, x1, y1 = (float(v) for v in bbox.split(","))
subset = subset.cx[x0:x1, y0:y1]
page = subset.iloc[:limit].copy()
page["geometry"] = shapely.set_precision(page.geometry.values, 1e-6) # fix 2
return Response(page.to_json(), media_type="application/geo+json",
headers={"ETag": etag,
"Cache-Control": "public, max-age=300",
"Vary": "Accept-Encoding"})
from fastapi.testclient import TestClient
client = TestClient(app)
first = client.get("/features?limit=50")
again = client.get("/features?limit=50", headers={"If-None-Match": first.headers["etag"]})
print(first.status_code, first.headers["content-encoding"], again.status_code,
client.get("/features?limit=1000").status_code)200 gzip 304 422
Note that the endpoint is def, not async def. FastAPI runs synchronous handlers in a thread pool, so GeoPandas work does not block the event loop — which is the correct default for a library that has no async interface.
Example 4 — the measurements in the table
import subprocess
import sys
SERVICE = """
import asyncio, hashlib, time
import geopandas as gpd
from fastapi import FastAPI, Request, Response
LAYER = gpd.read_file("provinces.gpkg")
app = FastAPI()
BODY = LAYER.iloc[:50].to_json() # one precomputed 50-feature page
ETAG = '"' + hashlib.sha256(BODY.encode()).hexdigest()[:16] + '"'
@app.get("/page")
def page(limit: int = 50): # serialised on every request
return Response(LAYER.iloc[:limit].to_json(), media_type="application/geo+json")
@app.get("/blocking")
async def blocking(): # 50 ms of synchronous work on the loop
time.sleep(0.05)
return Response(BODY, media_type="application/geo+json")
@app.get("/awaiting")
async def awaiting(): # the same 50 ms, awaited
await asyncio.sleep(0.05)
return Response(BODY, media_type="application/geo+json")
@app.get("/cached")
def cached(request: Request):
if request.headers.get("if-none-match") == ETAG:
return Response(status_code=304, headers={"ETag": ETAG})
return Response(BODY, media_type="application/geo+json", headers={"ETag": ETAG})
"""
open("bench_service.py", "w").write(SERVICE)
server = subprocess.Popen([sys.executable, "-m", "uvicorn", "bench_service:app", "--port", "8771",
"--log-level", "warning"])
BASE = "http://127.0.0.1:8771"
while True:
try:
httpx.get(f"{BASE}/cached", timeout=1)
break
except httpx.TransportError:
time.sleep(0.2)
print(f"one worker, load average {os.getloadavg()[0]:.1f}")
asyncio.run(load_test(f"{BASE}/page?limit=50", n=100))
asyncio.run(load_test(f"{BASE}/page?limit=1000", n=20))
asyncio.run(load_test(f"{BASE}/blocking"))
asyncio.run(load_test(f"{BASE}/awaiting"))
asyncio.run(load_test(f"{BASE}/cached"))
etag = httpx.get(f"{BASE}/cached").headers["etag"]
asyncio.run(load_test(f"{BASE}/cached", n=1000, headers={"If-None-Match": etag}))
server.terminate()
server.wait()
# payload levers, on the whole layer and on one page
full = layer.to_json()
rounded = layer.assign(geometry=shapely.set_precision(layer.geometry.values, 1e-6)).to_json()
print(f"whole layer {len(full) / 1e6:.2f} MB, coordinates at 1e-6 degrees {len(rounded) / 1e6:.2f} MB "
f"({1 - len(rounded) / len(full):.0%} smaller)")
import gzip
page50 = layer.iloc[:50].to_json().encode()
print(f"50-feature page {len(page50):,} bytes, gzip level 9 {len(gzip.compress(page50)):,} "
f"({len(page50) / len(gzip.compress(page50)):.2f}x)")one worker, load average 0.5
http://127.0.0.1:8771/page?limit=50
100 requests, 10 concurrent
23.1 rps p50 422.1 ms p95 482.9 ms
1,010,943 bytes per response, 23.3 MB/s sustained
http://127.0.0.1:8771/page?limit=1000
20 requests, 10 concurrent
2.6 rps p50 3465.5 ms p95 4313.4 ms
15,590,201 bytes per response, 40.1 MB/s sustained
http://127.0.0.1:8771/blocking
200 requests, 10 concurrent
19.8 rps p50 504.4 ms p95 505.5 ms
1,010,943 bytes per response, 20.0 MB/s sustained
http://127.0.0.1:8771/awaiting
200 requests, 10 concurrent
175.6 rps p50 54.1 ms p95 63.9 ms
1,010,943 bytes per response, 177.5 MB/s sustained
http://127.0.0.1:8771/cached
200 requests, 10 concurrent
586.4 rps p50 16.3 ms p95 19.3 ms
1,010,943 bytes per response, 592.8 MB/s sustained
http://127.0.0.1:8771/cached
1000 requests, 10 concurrent
1318.3 rps p50 5.3 ms p95 13.0 ms
0 bytes per response, 0.0 MB/s sustained
whole layer 68.35 MB, coordinates at 1e-6 degrees 46.44 MB (32% smaller)
50-feature page 1,010,943 bytes, gzip level 9 337,521 (3.00x)
Why one blocking call stalls everything
Why a blocking call is so much worse than it looks
A single worker's event loop is one thread. A 50 ms synchronous call does not slow that request by 50 ms — it stops every other request in that worker for 50 ms.
With ten concurrent clients the effect compounds into the measured 19.8 requests per second — exactly one per 50 ms — against 175.6 for the same work done without blocking. The fix is either an async library, run_in_threadpool, or removing the async keyword so FastAPI handles the threading.
Why payload size dominates spatial APIs
Most REST APIs return a few kilobytes. A spatial API returns geometry, and the measured spread is enormous: a 50-feature page at 1.01 MB, a 1,000-feature page at 15.59 MB, the unfiltered layer at 68.35 MB.
Serialisation is CPU work proportional to the output, and transfer is bandwidth proportional to the output. Both grow with the payload: 15.4 times the bytes cost 8.9 times the throughput, the difference being the fixed cost every request pays regardless of size.
That is also why a bigger machine helps less than expected: it does not make the response smaller.
Why caching beats every code optimisation
The measured 304 path served 1,318 requests per second because it did essentially nothing: compare a string, return a header. No query, no serialisation, no body.
No amount of query tuning approaches that, because the fastest possible query is still slower than not running one. And within a max-age window the request does not even arrive.
Caching is therefore the first thing to add and the last thing to remove, and its effectiveness is why response-size and blocking fixes should be judged on the uncached path only.
Why more workers is the last step
Workers multiply throughput and memory. A service holding a 500 MB layer in module scope costs 500 MB per worker; eight workers is 4 GB before any request is served.
They also do not fix the two largest problems. Eight workers each blocked on a synchronous call are eight blocked workers, and eight workers each serialising 15.6 MB are busy producing bytes nobody should have asked for. Fix the handler, then scale it.
Thread pools, per-worker data and sustained throughput
- A
defendpoint in FastAPI runs in a thread pool;async defdoes not. Choose deliberately. run_in_threadpoolis the escape hatch for synchronous work inside an async handler.- Module-level GeoDataFrames are per worker. Check memory before scaling out.
- Build
sindexat start-up, not on the first request. - Measure sustained MB/s, not just requests per second — it makes saturation obvious.
- Warm the service before load testing, or the first request's imports dominate.
--loop uvloop --http httptoolsis a small free improvement in uvicorn.- Judge fixes on the uncached path, then add the cache back.
Internal links
- Serving spatial data explained: files, features and tiles — sizing responses
- How to set cache headers on a spatial API and tile service — the largest single win
- How to paginate a large feature API in Python — bounding the payload
- How to add bounding box and attribute filters to a spatial API — reducing what is returned
- Fixing an API worker that runs out of memory — the other scaling failure
- How to rate limit and throttle a spatial API — bounding what clients may ask
- How to profile a slow Python GIS script — profiling the handler
- How to stream large query results from a spatial API — for genuinely large results
FAQ
Why is my API fast in testing and slow under load?
Usually one of four things: a blocking call in an async endpoint, an oversized response, no caching, or a single worker. Measured, the first three were worth 8.9×, 8.9× and 25× (plus 2.2× for a 304); the fourth is worth at most the worker count.
How much does a blocking call cost?
Measured with a 50 ms synchronous call inside async def: 19.8 requests per second against 175.6 for the same wait awaited.
Should I add more workers?
Last. Eight workers each blocked, or each serialising 15.6 MB, are eight times the problem. Fix the handler first, and check memory — module-level data is per worker.
What is the single biggest improvement?
Caching. Serialising a 50-feature page on every request ran at 23.1 requests per second; a stored body at 586.4, and a conditional request answered with 304 at 1,318.3.
How do I know whether the payload is the problem?
Compare throughput at two page sizes. Measured, 50 features gave 23.1 requests per second and 1,000 gave 2.6, and the larger page sustained 40.1 MB/s: the bytes are the work.
Is async def always better?
No. FastAPI runs plain def handlers in a thread pool, which is the right choice for synchronous libraries such as GeoPandas. async def is only better when the work is genuinely awaitable.