🔧 Fix APIs & tile services Performance & scale Serving Spatial Data · step 20 Notebook By · Updated

Fixing an API Worker That Runs Out of Memory

On this page

Problem statement

The worker is killed. Sometimes there is an exception; more often the process simply disappears and the supervisor restarts it:

Worker (pid:1247) was sent SIGKILL! Perhaps out of memory?

For a spatial API there are four distinct causes, and they need different fixes:

  • The layer is loaded per worker. A 500 MB GeoDataFrame in module scope costs 500 MB × the worker count. Eight workers is 4 GB before a request arrives.
  • A single response is enormous. Measured on Natural Earth's 4,596 provinces, an unfiltered feature response was 68.35 MB and a 1,000-feature page was 15.6 MB — and the body is built in memory before it is sent.
  • Concurrent large requests multiply. Ten clients each triggering a 15.6 MB response is 156 MB of response bodies alive at once, plus the intermediate structures that produced them.
  • Something accumulates. A cache with no bound, a list of results, a connection pool that grows.

The first two are architecture; the last two are bugs. Telling them apart takes one measurement each.

Quick answer

import resource
import os


def memory_report(label=""):
    peak_mb = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss / 1024
    with open("/proc/self/status") as status:
        current = next((line for line in status if line.startswith("VmRSS")), "")
    print(f"{label:24} peak {peak_mb:8,.0f} MB   {current.strip()}")


@app.on_event("startup")
def on_startup():
    memory_report("after loading layers")      # this × workers is your floor

Then the four fixes, in order of how often they apply:

# 1. do not hold the data per worker
LAYER = None                                    # load from a database or a file per request

# 2. bound every response
limit: int = Query(50, ge=1, le=200)

# 3. stream anything genuinely large
return StreamingResponse(generate(), media_type="application/geo+json-seq")

# 4. bound concurrency on expensive endpoints
HEAVY = asyncio.Semaphore(4)
Four causes of an API worker running out of memory. two architecture, two bugscausefixthe layer is loaded per workera database, per-request reads, orfewer workersone response is enormouscap the page size — a body isbuilt in memoryconcurrent large requestsa semaphore on expensive endpointssomething accumulatestracemalloc — usually an unboundedcacheA SIGKILL with no traceback is the OS out-of-memory killer, not an application error spatialworkflow.io
Measure the baseline per worker and multiply — that number is the floor.

Baseline, shared data, bounded responses, streaming

1. Measure the per-worker baseline

The floor for the whole service is the memory one worker uses after start-up, multiplied by the worker count. Print it at start-up and multiply:

after loading layers     peak      284 MB   VmRSS:	  283964 kB

That is the server on this page after loading one 4,596-polygon layer. Eight workers of it is 2.3 GB, idle, before a single response has been built. On a 2 GB container the service is dead before the first request, and no request-level optimisation helps.

2. Stop holding the data per worker

Three ways out, in increasing order of effort:

  • Query a database instead. PostGIS or a DuckDB file holds the data once, outside the workers.
  • Read from a file per request. A Parquet or FlatGeobuf file with a spatial filter reads only what is needed; the operating system's page cache is shared between workers.
  • Use fewer workers with more threads. Synchronous FastAPI handlers run in a thread pool, so one worker with sixteen threads serves concurrent requests without duplicating the data.

The third is frequently the quickest fix and it is genuinely effective for I/O-bound handlers.

3. Bound the response, because the body is built in memory

to_json() produces the entire string before it is sent. Measured, building a 15.6 MB body of 1,000 features peaked at 67.7 MB of Python allocations — 4.3 times the final size, alive at once.

Ten concurrent such requests is not 156 MB but closer to 680 MB. A page-size cap is therefore a memory control as much as a performance one:

limit: int = Query(50, ge=1, le=200)

4. Stream what genuinely has to be large

A streamed response holds one chunk at a time:

def generate():
    for start in range(0, len(subset), 500):
        page = subset.iloc[start:start + 500]
        for feature in json.loads(page.to_json())["features"]:
            yield json.dumps(feature) + "\n"

return StreamingResponse(generate(), media_type="application/geo+json-seq")

The peak becomes a property of the chunk size rather than of the result. The laziness has to reach the data source, though: a generator whose first line calls .df() has streamed nothing.

5. Bound the concurrency of expensive endpoints

A rate limit bounds arrivals; a semaphore bounds how many expensive requests are in flight at once:

import asyncio
from fastapi import HTTPException

HEAVY = asyncio.Semaphore(4)


@app.get("/download")
async def download():
    try:
        await asyncio.wait_for(HEAVY.acquire(), timeout=0.5)
    except asyncio.TimeoutError:
        raise HTTPException(503, "too many exports in progress",
                            headers={"Retry-After": "30"})
    try:
        return await build_export()
    finally:
        HEAVY.release()

Four concurrent 15.6 MB responses is 62 MB of bodies and, at 4.3 times, about 270 MB at the peak; forty is 2.7 GB and a dead worker.

6. Find the leak, if memory grows rather than spikes

A spike is a big request; growth is a leak. The difference is visible in a graph and diagnosable with tracemalloc:

import tracemalloc

tracemalloc.start(10)
baseline = tracemalloc.take_snapshot()

# ... serve some requests ...

current = tracemalloc.take_snapshot()
for stat in current.compare_to(baseline, "lineno")[:10]:
    print(stat)

The usual culprits in a spatial API: an unbounded functools.lru_cache on a function returning GeoDataFrames, a module-level list appended per request, and a connection pool with no maximum.

Module-level data multiplied by the worker count. a 284 MB baseline, by worker count1 worker284 MB2 workers568 MB4 workers1.1 GB8 workers2.3 GB — idle02400 MBCopy-on-write does not save you: reference counting dirties the page it touches spatialworkflow.io
On a 2 GB container, the service is dead before the first request.

Memory-aware start-up and per-request reads

The data used on this page

The layer is Natural Earth's 1:10m admin-1 states and provinces (4,596 polygons, 121 attributes), as provinces.gpkg and provinces.parquet. The setup writes oom_server.py — a FastAPI app that loads the whole layer at module scope, the pattern this page warns about, with the /health endpoint from Example 1 — and starts one uvicorn worker on port 8765. Every measurement on the page is taken against that process.

import atexit
import os
import subprocess
import sys
import time

import geopandas as gpd
import httpx

if not os.path.exists("provinces.gpkg"):
    gpd.read_file("https://naciscdn.org/naturalearth/10m/cultural/ne_10m_admin_1_states_provinces.zip") \
       .to_file("provinces.gpkg")
if not os.path.exists("provinces.parquet"):
    gpd.read_file("provinces.gpkg").to_parquet("provinces.parquet")

with open("oom_server.py", "w") as f:
    f.write('''
import os
import resource

import geopandas as gpd
from fastapi import FastAPI, Query, Response


def memory_report(label=""):
    peak_mb = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss / 1024
    with open("/proc/self/status") as status:
        current = next((line for line in status if line.startswith("VmRSS")), "")
    print(f"{label:24} peak {peak_mb:8,.0f} MB   {current.strip()}", flush=True)


def rss_mb():
    with open("/proc/self/statm") as handle:
        pages = int(handle.read().split()[1])
    return pages * os.sysconf("SC_PAGE_SIZE") / 1e6


LAYER = gpd.read_file("provinces.gpkg")          # module scope: one copy per worker
app = FastAPI()
BASELINE = rss_mb()
memory_report("after loading layers")


@app.get("/features")
def features(limit: int = Query(50, ge=1, le=5000)):
    return Response(LAYER.iloc[:limit].to_json(), media_type="application/geo+json")


@app.get("/health")
def health():
    current = rss_mb()
    return {"rss_mb": round(current), "growth_mb": round(current - BASELINE),
            "peak_mb": round(resource.getrusage(resource.RUSAGE_SELF).ru_maxrss / 1024)}
''')

BASE = "http://127.0.0.1:8765"
server = subprocess.Popen([sys.executable, "-m", "uvicorn", "oom_server:app", "--port", "8765",
                           "--workers", "1", "--log-level", "warning"],
                          stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True)
atexit.register(server.terminate)
startup_line = server.stdout.readline().rstrip()
for _ in range(120):
    try:
        httpx.get(f"{BASE}/health")
        break
    except httpx.ConnectError:
        time.sleep(0.5)
print(startup_line)
after loading layers     peak      284 MB   VmRSS:	  283964 kB

Example 1 — a memory-aware start-up and health check

import os
import resource
from fastapi import FastAPI

app = FastAPI()


def rss_mb() -> float:
    with open("/proc/self/statm") as handle:
        pages = int(handle.read().split()[1])
    return pages * os.sysconf("SC_PAGE_SIZE") / 1e6


@app.on_event("startup")
def report_baseline():
    app.state.baseline_mb = rss_mb()
    workers = int(os.environ.get("WEB_CONCURRENCY", "1"))
    print(f"baseline {app.state.baseline_mb:,.0f} MB per worker; "
          f"{workers} workers → {app.state.baseline_mb * workers:,.0f} MB floor")


@app.get("/health")
def health():
    current = rss_mb()
    growth = current - app.state.baseline_mb
    return {"status": "ok" if growth < 500 else "degraded",
            "rss_mb": round(current),
            "growth_mb": round(growth),
            "peak_mb": round(resource.getrusage(resource.RUSAGE_SELF).ru_maxrss / 1024)}

Exposing growth in the health check is what turns "the workers keep restarting" into a graph with a slope.

Example 2 — reading per request instead of holding the layer

import geopandas as gpd
from fastapi import FastAPI, Query, Response

app = FastAPI()
LAYER_PATH = "provinces.parquet"


@app.get("/features")
def features(bbox: str | None = None, limit: int = Query(50, ge=1, le=200)):
    """Read only the rows the request needs. The OS page cache is shared
    between workers; a module-level GeoDataFrame is not."""
    if bbox:
        x0, y0, x1, y1 = (float(v) for v in bbox.split(","))
        subset = gpd.read_parquet(LAYER_PATH, bbox=(x0, y0, x1, y1))
    else:
        subset = gpd.read_parquet(LAYER_PATH, columns=["name", "admin", "geometry"])

    return Response(subset.iloc[:limit].to_json(),
                    media_type="application/geo+json")

GeoParquet's row-group statistics mean a bounding-box read touches a fraction of the file. In Columnar vs row storage, a selective query against a 446 MB Parquet file transferred 0.26 MB — the same principle applies to a local read.

Example 3 — a load test that watches memory

import asyncio
import httpx


async def memory_under_load(base_url, path, rounds=5, n=100, concurrency=10):
    """Growth across rounds is a leak; a flat line with spikes is not."""
    async with httpx.AsyncClient(timeout=60) as client:
        for round_number in range(1, rounds + 1):
            limits = httpx.Limits(max_connections=concurrency)
            async with httpx.AsyncClient(limits=limits, timeout=60) as worker:
                semaphore = asyncio.Semaphore(concurrency)

                async def one():
                    async with semaphore:
                        await worker.get(f"{base_url}{path}")

                await asyncio.gather(*[one() for _ in range(n)])

            health = (await client.get(f"{base_url}/health")).json()
            print(f"round {round_number}: rss {health['rss_mb']:,} MB "
                  f"(+{health['growth_mb']:,} from baseline), "
                  f"peak {health['peak_mb']:,} MB")


asyncio.run(memory_under_load(BASE, "/features?limit=1000", rounds=3))
round 1: rss 932 MB (+641 from baseline), peak 970 MB
round 2: rss 948 MB (+658 from baseline), peak 971 MB
round 3: rss 959 MB (+668 from baseline), peak 983 MB

Round 1 raised resident memory by 641 MB and it stayed there; rounds 2 and 3 added 17 and 10 MB. One step up and then a plateau is the process keeping the memory that ten concurrent 15.6 MB responses needed at once (Example 5). A leak climbs by a similar amount every round.

Example 4 — the measurements on this page

import asyncio
import time
import tracemalloc

for limit in (50, 1000, 5000):
    reply = httpx.get(f"{BASE}/features", params={"limit": limit}, timeout=300)
    print(f"limit={limit}: {len(reply.json()['features']):,} features, {len(reply.content) / 1e6:.2f} MB")

layer = gpd.read_file("provinces.gpkg")
tracemalloc.start()
body = layer.iloc[:1000].to_json()
_, peak = tracemalloc.get_traced_memory()
tracemalloc.stop()
print(f"building a 1,000-feature body of {len(body) / 1e6:.1f} MB peaked at "
      f"{peak / 1e6:.1f} MB of Python allocations ({peak / len(body):.1f}x)")


async def rate(limit, n=40, concurrency=10):
    async with httpx.AsyncClient(timeout=300, limits=httpx.Limits(max_connections=concurrency)) as client:
        gate = asyncio.Semaphore(concurrency)

        async def one():
            async with gate:
                await client.get(f"{BASE}/features", params={"limit": limit})

        start = time.perf_counter()
        await asyncio.gather(*[one() for _ in range(n)])
        return n / (time.perf_counter() - start)

print(f"load average {os.getloadavg()[0]:.1f}")
for limit in (50, 1000):
    print(f"limit={limit}, 10 concurrent: {asyncio.run(rate(limit)):.1f} requests/s")
print("after everything:", httpx.get(f"{BASE}/health").json())
server.terminate()
server.wait()
limit=50: 50 features, 1.01 MB
limit=1000: 1,000 features, 15.59 MB
limit=5000: 4,596 features, 68.35 MB
building a 1,000-feature body of 15.6 MB peaked at 67.7 MB of Python allocations (4.3x)
load average 2.2
limit=50, 10 concurrent: 21.1 requests/s
limit=1000, 10 concurrent: 2.5 requests/s
after everything: {'rss_mb': 1007, 'growth_mb': 717, 'peak_mb': 1064}
Resident memory growth under three rounds of 100 requests for 1,000-feature pages, with and without capped malloc arenas. growth above the 284 MB baselineround 1round 2round 3default glibc+641 MB+658 MB+668 MBMALLOC_ARENA_MAX=2+504 MB+506 MB+479 MBthe same burst in-process: rss +610 MB, Python objects still alive 3.1 MBOne step, then a plateau: memory kept by the allocator, not held by the code spatialworkflow.io
Round 1 sets the plateau; a leak would add a similar step every round.

Example 5 — is the plateau a leak?

from concurrent.futures import ThreadPoolExecutor


def rss_mb():
    with open("/proc/self/statm") as handle:
        return int(handle.read().split()[1]) * os.sysconf("SC_PAGE_SIZE") / 1e6


# The same work as the server's burst, in this process, with Python's own allocations traced
before = rss_mb()
tracemalloc.start()
with ThreadPoolExecutor(10) as pool:
    list(pool.map(lambda _: len(layer.iloc[:1000].to_json()), range(100)))
live, _ = tracemalloc.get_traced_memory()
tracemalloc.stop()
print(f"after 100 bodies on 10 threads: rss +{rss_mb() - before:,.0f} MB, "
      f"Python objects still alive {live / 1e6:.1f} MB")

# The server again, with glibc limited to two malloc arenas
capped = subprocess.Popen([sys.executable, "-m", "uvicorn", "oom_server:app", "--port", "8765",
                           "--log-level", "warning"], env=dict(os.environ, MALLOC_ARENA_MAX="2"),
                          stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True)
capped.stdout.readline()
for _ in range(120):
    try:
        httpx.get(f"{BASE}/health")
        break
    except httpx.ConnectError:
        time.sleep(0.5)
print("MALLOC_ARENA_MAX=2:")
asyncio.run(memory_under_load(BASE, "/features?limit=1000", rounds=3))
capped.terminate()
capped.wait()
after 100 bodies on 10 threads: rss +610 MB, Python objects still alive 3.1 MB
MALLOC_ARENA_MAX=2:
round 1: rss 795 MB (+504 from baseline), peak 3,174 MB
round 2: rss 797 MB (+506 from baseline), peak 3,174 MB
round 3: rss 769 MB (+479 from baseline), peak 3,174 MB

Python itself was holding 3.1 MB of live objects while resident memory stood 610 MB higher than before: nothing leaked, but the freed memory stayed with the process instead of going back to the operating system. Limiting glibc to two arenas lowered the plateau to about 500 MB, and in this run the recorded peak rose to 3.2 GB — a trade, not a fix. What bounds the plateau is how many large bodies can be built at once: the page cap and the semaphore above.

Why module-level data multiplies

Why module-level data multiplies

Each uvicorn or gunicorn worker is a separate process with its own interpreter and its own copy of everything imported at module scope. A 500 MB GeoDataFrame loaded at import is loaded once per worker.

Copy-on-write after a fork helps briefly and stops helping quickly, because Python's reference counting writes to object headers — touching an object dirties its page. Practically, assume no sharing.

The fixes are to move the data out of the process, or to use fewer processes with more threads.

Why the peak is several times the response size

Producing a 15.6 MB GeoJSON response involves the source GeoDataFrame slice, an intermediate representation, the JSON string, and the response body — several copies alive at the same moment. Measured, the peak was 4.3 times the body.

That is why a page-size cap is a memory control. The measured throughput difference between 1.01 MB pages of 50 features and 15.6 MB pages of 1,000 was 21.1 against 2.5 requests per second with ten concurrent clients (load average 2.2), and the memory difference is proportionally worse under concurrency, because ten of them coexist.

Why streaming changes the class of the problem

A streamed response never has a whole body. The generator yields chunks and the server writes them, so the peak is a chunk plus whatever produced it.

That converts memory from a function of the result to a function of the chunk size, which is a constant you choose. It is the only fix that works when the result genuinely has to be large.

Why a semaphore and a rate limit are different controls

A rate limit says how often a caller may ask. A semaphore says how many expensive operations may be in progress at once, across all callers.

Ten different clients each making one legitimate request at the same moment do not breach any rate limit, and if each request costs 100 MB the worker dies. Concurrency is the dimension that matters for memory, and it needs its own bound.

Telling a big-response spike from a memory leak. run several rounds and watch the floorspike✕one step up, then a plateau✕Python objects live: a few MB✕set by concurrent body size✕fix: cap the page, add a semaphoreleak✕climbs a similar step every round✕Python objects grow with it✕caused by accumulation✕fix: tracemalloc, find the cacheOne step and a plateau is memory kept after a burst; a step every round is a leak spatialworkflow.io
The usual leak is an lru_cache on a function returning GeoDataFrames.

Units, the OOM killer and container limits

  • ru_maxrss is kilobytes on Linux and bytes on macOS.
  • A SIGKILL with no traceback is the OS out-of-memory killer, not an application error.
  • Container memory limits are invisible to Python — the process sees the host's RAM.
  • Copy-on-write does not save you — reference counting dirties pages.
  • --max-requests in gunicorn recycles workers and hides a leak rather than fixing it.
  • gc.freeze() after loading static data before forking can help a little.
  • A lru_cache on a function returning GeoDataFrames is an unbounded memory sink.
  • Measure the baseline per worker and multiply. That number is the floor.

FAQ

Why does my API worker get killed with no traceback?

That is the operating system's out-of-memory killer. Python never sees it, so there is no exception — only a supervisor restarting the process.

Why does memory scale with the worker count?

Because each worker is a separate process with its own copy of everything imported at module scope. A 500 MB layer costs 500 MB per worker, and copy-on-write does not save you.

How much memory does one response need?

About four times its size. A 15.6 MB GeoJSON response peaked at 67.7 MB of allocations while it was built — and ten concurrent ones coexist.

Should I use more workers or more threads?

More threads, for I/O-bound synchronous handlers: FastAPI runs plain def endpoints in a thread pool, and threads share the data that processes duplicate.

How do I tell a leak from a spike?

Run several rounds of load and watch the resident size between them. One step up and then a plateau is memory kept after a burst of big responses; climbing every round is a leak.

What is the quickest fix?

Cap the page size and reduce the worker count. Both take one line, and together they address the two causes that account for most of these failures.