🔧 Fix APIs & tile services Performance & scale Serving Spatial Data · step 20 of 20 Updated

Fixing an API Worker That Runs Out of Memory

Problem statement

The worker is killed. Sometimes there is an exception; more often the process simply disappears and the supervisor restarts it:

Worker (pid:1247) was sent SIGKILL! Perhaps out of memory?

For a spatial API there are four distinct causes, and they need different fixes:

  • The layer is loaded per worker. A 500 MB GeoDataFrame in module scope costs 500 MB × the worker count. Eight workers is 4 GB before a request arrives.
  • A single response is enormous. Measured, an unfiltered feature response was 54 MB and a 1,000-feature page was 12.4 MB — and the body is built in memory before it is sent.
  • Concurrent large requests multiply. Ten clients each triggering a 12.4 MB response is 124 MB of response bodies alive at once, plus the intermediate structures that produced them.
  • Something accumulates. A cache with no bound, a list of results, a connection pool that grows.

The first two are architecture; the last two are bugs. Telling them apart takes one measurement each.

Quick answer 

import resource
import os


def memory_report(label=""):
    peak_mb = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss / 1024
    with open("/proc/self/status") as status:
        current = next((line for line in status if line.startswith("VmRSS")), "")
    print(f"{label:24} peak {peak_mb:8,.0f} MB   {current.strip()}")


@app.on_event("startup")
def on_startup():
    memory_report("after loading layers")      # this × workers is your floor

Then the four fixes, in order of how often they apply:

# 1. do not hold the data per worker
LAYER = None                                    # load from a database or a file per request

# 2. bound every response
limit: int = Query(50, ge=1, le=200)

# 3. stream anything genuinely large
return StreamingResponse(generate(), media_type="application/geo+json-seq")

# 4. bound concurrency on expensive endpoints
HEAVY = asyncio.Semaphore(4)
Triage table of four causes of API worker memory exhaustion and their fixes.
Measure the baseline per worker and multiply — that number is the floor.

Step-by-step solution

1. Measure the per-worker baseline

The floor for the whole service is the memory one worker uses after start-up, multiplied by the worker count. Print it at start-up and multiply:

after loading layers      peak      612 MB   VmRSS: 587,224 kB

Eight workers of that is 4.7 GB, idle. If the container's limit is 4 GB, the service is dead before the first request and no request-level optimisation helps.

2. Stop holding the data per worker

Three ways out, in increasing order of effort:

  • Query a database instead. PostGIS or a DuckDB file holds the data once, outside the workers.
  • Read from a file per request. A Parquet or FlatGeobuf file with a spatial filter reads only what is needed; the operating system's page cache is shared between workers.
  • Use fewer workers with more threads. Synchronous FastAPI handlers run in a thread pool, so one worker with sixteen threads serves concurrent requests without duplicating the data.

The third is frequently the quickest fix and it is genuinely effective for I/O-bound handlers.

3. Bound the response, because the body is built in memory

to_json() produces the entire string before it is sent. A 12.4 MB response is 12.4 MB of Python string, plus the intermediate GeoDataFrame, plus the serialisation buffers — usually two to three times the final size, alive at once.

Ten concurrent such requests is not 124 MB but several hundred. A page-size cap is therefore a memory control as much as a performance one:

limit: int = Query(50, ge=1, le=200)

4. Stream what genuinely has to be large

A streamed response holds one chunk at a time:

def generate():
    for start in range(0, len(subset), 500):
        page = subset.iloc[start:start + 500]
        for feature in json.loads(page.to_json())["features"]:
            yield json.dumps(feature) + "\n"

return StreamingResponse(generate(), media_type="application/geo+json-seq")

The peak becomes a property of the chunk size rather than of the result. The laziness has to reach the data source, though: a generator whose first line calls .df() has streamed nothing.

5. Bound the concurrency of expensive endpoints

A rate limit bounds arrivals; a semaphore bounds how many expensive requests are in flight at once:

import asyncio
from fastapi import HTTPException

HEAVY = asyncio.Semaphore(4)


@app.get("/download")
async def download():
    try:
        await asyncio.wait_for(HEAVY.acquire(), timeout=0.5)
    except asyncio.TimeoutError:
        raise HTTPException(503, "too many exports in progress",
                            headers={"Retry-After": "30"})
    try:
        return await build_export()
    finally:
        HEAVY.release()

Four concurrent 12.4 MB exports is 50 MB of bodies; forty is 500 MB and a dead worker.

6. Find the leak, if memory grows rather than spikes

A spike is a big request; growth is a leak. The difference is visible in a graph and diagnosable with tracemalloc:

import tracemalloc

tracemalloc.start(10)
baseline = tracemalloc.take_snapshot()

# ... serve some requests ...

current = tracemalloc.take_snapshot()
for stat in current.compare_to(baseline, "lineno")[:10]:
    print(stat)

The usual culprits in a spatial API: an unbounded functools.lru_cache on a function returning GeoDataFrames, a module-level list appended per request, and a connection pool with no maximum.

Bar chart of total memory for a 612 MB baseline across one to eight workers.
If the container limit is 4 GB, the service is dead before the first request.

Code examples

Example 1 — a memory-aware start-up and health check 

import os
import resource
from fastapi import FastAPI

app = FastAPI()


def rss_mb() -> float:
    with open("/proc/self/statm") as handle:
        pages = int(handle.read().split()[1])
    return pages * os.sysconf("SC_PAGE_SIZE") / 1e6


@app.on_event("startup")
def report_baseline():
    app.state.baseline_mb = rss_mb()
    workers = int(os.environ.get("WEB_CONCURRENCY", "1"))
    print(f"baseline {app.state.baseline_mb:,.0f} MB per worker; "
          f"{workers} workers → {app.state.baseline_mb * workers:,.0f} MB floor")


@app.get("/health")
def health():
    current = rss_mb()
    growth = current - app.state.baseline_mb
    return {"status": "ok" if growth < 500 else "degraded",
            "rss_mb": round(current),
            "growth_mb": round(growth),
            "peak_mb": round(resource.getrusage(resource.RUSAGE_SELF).ru_maxrss / 1024)}

Exposing growth in the health check is what turns "the workers keep restarting" into a graph with a slope.

Example 2 — reading per request instead of holding the layer 

import geopandas as gpd
from fastapi import FastAPI, Query, Response

app = FastAPI()
LAYER_PATH = "provinces.parquet"


@app.get("/features")
def features(bbox: str | None = None, limit: int = Query(50, ge=1, le=200)):
    """Read only the rows the request needs. The OS page cache is shared
    between workers; a module-level GeoDataFrame is not."""
    if bbox:
        x0, y0, x1, y1 = (float(v) for v in bbox.split(","))
        subset = gpd.read_parquet(LAYER_PATH, bbox=(x0, y0, x1, y1))
    else:
        subset = gpd.read_parquet(LAYER_PATH, columns=["name", "admin", "geometry"])

    return Response(subset.iloc[:limit].to_json(),
                    media_type="application/geo+json")

GeoParquet's row-group statistics mean a bounding-box read touches a fraction of the file. Measured elsewhere on a 446 MB Parquet file, a selective query transferred 0.26 MB — the same principle applies to a local read.

Example 3 — a load test that watches memory 

import asyncio
import httpx


async def memory_under_load(base_url, path, rounds=5, n=100, concurrency=10):
    """Growth across rounds is a leak; a flat line with spikes is not."""
    async with httpx.AsyncClient(timeout=60) as client:
        for round_number in range(1, rounds + 1):
            limits = httpx.Limits(max_connections=concurrency)
            async with httpx.AsyncClient(limits=limits, timeout=60) as worker:
                semaphore = asyncio.Semaphore(concurrency)

                async def one():
                    async with semaphore:
                        await worker.get(f"{base_url}{path}")

                await asyncio.gather(*[one() for _ in range(n)])

            health = (await client.get(f"{base_url}/health")).json()
            print(f"round {round_number}: rss {health['rss_mb']:,} MB "
                  f"(+{health['growth_mb']:,} from baseline), "
                  f"peak {health['peak_mb']:,} MB")
round 1: rss 742 MB (+130 from baseline), peak 981 MB
round 2: rss 748 MB (+136 from baseline), peak 981 MB
round 3: rss 751 MB (+139 from baseline), peak 981 MB

Flat growth with a high peak is a big-response problem. Growth that climbs every round is a leak.

Explanation

Why module-level data multiplies

Each uvicorn or gunicorn worker is a separate process with its own interpreter and its own copy of everything imported at module scope. A 500 MB GeoDataFrame loaded at import is loaded once per worker.

Copy-on-write after a fork helps briefly and stops helping quickly, because Python's reference counting writes to object headers — touching an object dirties its page. Practically, assume no sharing.

The fixes are to move the data out of the process, or to use fewer processes with more threads.

Why the peak is several times the response size

Producing a 12.4 MB GeoJSON response involves the source GeoDataFrame slice, an intermediate representation, the JSON string, and the response body — several copies alive at the same moment.

That is why a page-size cap is a memory control. The measured throughput difference between 857 kB and 12.4 MB pages was 32.5 against 2.3 requests per second, and the memory difference is proportionally worse under concurrency, because ten of them coexist.

Why streaming changes the class of the problem

A streamed response never has a whole body. The generator yields chunks and the server writes them, so the peak is a chunk plus whatever produced it.

That converts memory from a function of the result to a function of the chunk size, which is a constant you choose. It is the only fix that works when the result genuinely has to be large.

Why a semaphore and a rate limit are different controls

A rate limit says how often a caller may ask. A semaphore says how many expensive operations may be in progress at once, across all callers.

Ten different clients each making one legitimate request at the same moment do not breach any rate limit, and if each request costs 100 MB the worker dies. Concurrency is the dimension that matters for memory, and it needs its own bound.

Two panels distinguishing a memory spike from a memory leak under load.
The usual leak is an lru_cache on a function returning GeoDataFrames.

Edge cases or notes

  • ru_maxrss is kilobytes on Linux and bytes on macOS.
  • A SIGKILL with no traceback is the OS out-of-memory killer, not an application error.
  • Container memory limits are invisible to Python — the process sees the host's RAM.
  • Copy-on-write does not save you — reference counting dirties pages.
  • --max-requests in gunicorn recycles workers and hides a leak rather than fixing it.
  • gc.freeze() after loading static data before forking can help a little.
  • A lru_cache on a function returning GeoDataFrames is an unbounded memory sink.
  • Measure the baseline per worker and multiply. That number is the floor.

FAQ

Why does my API worker get killed with no traceback?

That is the operating system's out-of-memory killer. Python never sees it, so there is no exception — only a supervisor restarting the process.

Why does memory scale with the worker count?

Because each worker is a separate process with its own copy of everything imported at module scope. A 500 MB layer costs 500 MB per worker, and copy-on-write does not save you.

How much memory does one response need?

Several times its size. A 12.4 MB GeoJSON response involves the source slice, an intermediate form and the final string, all alive at once — and ten concurrent ones coexist.

Should I use more workers or more threads?

More threads, for I/O-bound synchronous handlers: FastAPI runs plain def endpoints in a thread pool, and threads share the data that processes duplicate.

How do I tell a leak from a spike?

Run several rounds of load and watch the resident size between them. Flat with a high peak is a big-response problem; climbing every round is a leak.

What is the quickest fix?

Cap the page size and reduce the worker count. Both take one line, and together they address the two causes that account for most of these failures.