⚙️ How-To APIs & tile services Serving Spatial Data · step 14 of 20 Updated

How to Rate Limit and Throttle a Spatial API

Problem statement

Rate limiting a spatial API is not the same problem as rate limiting a REST API, because the requests are not comparable. Measured on one service:

request                       payload      throughput
one vector tile               ~20 kB        hundreds/s
50-feature page              857 kB          32.5 rps
1,000-feature page          12.43 MB          2.3 rps
whole dataset               54.06 MB          ~2 rps

A limit of "100 requests per minute" either strangles a map that legitimately needs hundreds of tiles, or permits a hundred 54 MB downloads. The two are three orders of magnitude apart, and no single request count is right for both.

The second complication is operational: a limiter that keeps its state in a process is not a limiter when there are eight workers.

Quick answer

Weight by cost, share the state, and always send Retry-After:

import time
from fastapi import HTTPException

COST = {"tile": 1.0, "features": 5.0, "download": 50.0}
BUDGETS = {"public": (300, 60), "partner": (3000, 60)}     # units, per seconds


class TokenBucket:
    def __init__(self, store):
        self.store = store                # Redis in production

    def consume(self, key, capacity, per_seconds, cost=1.0):
        rate = capacity / per_seconds
        tokens, updated = self.store.get(key, (capacity, time.monotonic()))
        now = time.monotonic()
        tokens = min(capacity, tokens + (now - updated) * rate)

        if tokens < cost:
            wait = (cost - tokens) / rate
            raise HTTPException(429, "rate limit exceeded", headers={
                "Retry-After": str(int(wait) + 1),
                "X-RateLimit-Limit": str(int(capacity)),
                "X-RateLimit-Remaining": "0",
                "X-RateLimit-Reset": str(int(time.time() + wait))})

        self.store.set(key, (tokens - cost, now))
        return int(tokens - cost)

A token bucket allows a burst up to the capacity and then settles to the average rate, which matches how map clients behave: a pan requests twenty tiles at once and then nothing for ten seconds.

Grid of three rate-limiting algorithms with their behaviour and best fit.
Set the bucket capacity to at least one screenful of tiles.

Step-by-step solution

1. Price the endpoints

Assign each endpoint a cost roughly proportional to the work it causes. Approximate is fine; the point is to stop treating a tile and a bulk download as equivalent.

COST = {
    "GET /tiles/{z}/{x}/{y}": 1.0,
    "GET /features":          5.0,
    "GET /features/stream":  20.0,
    "GET /download":         50.0,
}

A useful calibration: make one unit approximately the cheapest request, then scale from measured payloads and timings. The measured ratio between a tile and a 1,000-feature page justifies a factor of about fifty.

2. Choose the algorithm from the traffic shape

  • Token bucket — allows bursts up to a capacity, then the average rate. Right for map clients, which burst by nature.
  • Sliding window — a strict count over the last N seconds. Fairer, and it punishes the burst that a map tile load legitimately produces.
  • Leaky bucket — a constant drain rate. Right for protecting a fragile downstream service.

For a tile endpoint the token bucket is nearly always the right choice, with a capacity large enough for one screenful of tiles.

3. Share the state, or the limit is a lie

Eight uvicorn workers with a process-local dictionary enforce eight times the intended limit, distributed unpredictably. Use Redis:

import redis

LUA = """
local tokens_key = KEYS[1]
local capacity = tonumber(ARGV[1])
local rate = tonumber(ARGV[2])
local now = tonumber(ARGV[3])
local cost = tonumber(ARGV[4])

local data = redis.call('HMGET', tokens_key, 'tokens', 'updated')
local tokens = tonumber(data[1]) or capacity
local updated = tonumber(data[2]) or now
tokens = math.min(capacity, tokens + (now - updated) * rate)

if tokens < cost then
  return {0, tostring(tokens)}
end
tokens = tokens - cost
redis.call('HMSET', tokens_key, 'tokens', tokens, 'updated', now)
redis.call('EXPIRE', tokens_key, 3600)
return {1, tostring(tokens)}
"""

The Lua script makes the read-modify-write atomic, which a Python round trip does not.

4. Key by caller, with a sensible fallback 

def limit_key(request) -> tuple[str, str]:
    if (key := request.headers.get("x-api-key")):
        return f"key:{hash_key(key)}", "partner"
    forwarded = request.headers.get("x-forwarded-for", "")
    ip = forwarded.split(",")[0].strip() or request.client.host
    return f"ip:{ip}", "public"

Trust X-Forwarded-For only behind a proxy you control, and take the first entry. Remember that an IP can be an entire office, so the anonymous budget should be generous enough for a shared address and small enough to bound abuse.

5. Send the headers that let clients behave 

429 Too Many Requests
Retry-After: 12
X-RateLimit-Limit: 300
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 1757068812

Retry-After is the one that matters. Without it a client with a retry loop turns one rejection into a hundred, which is the exact traffic pattern the limit exists to prevent.

Sending X-RateLimit-Remaining on successful responses lets a well-behaved client slow itself down before being rejected.

6. Exempt the cheap paths

A 304 response costs almost nothing — measured at 1,191 requests per second and 6.2 ms, against 100 requests per second for generating the body. Charging it the same as a full response penalises exactly the clients that are caching properly.

Charge on the way out, based on what was actually done:

cost = 0.1 if response.status_code == 304 else COST[endpoint]
Two panels contrasting a process-local rate limiter with a shared Redis one.
The cost is one round trip against the tens of milliseconds a spatial request already takes.

Code examples

Example 1 — middleware with cost weighting and Redis state 

import hashlib
import time
import redis
from fastapi import FastAPI, Request
from fastapi.responses import JSONResponse

app = FastAPI()
pool = redis.ConnectionPool.from_url("redis://localhost:6379/0")

COST = {"tiles": 1.0, "features": 5.0, "stream": 20.0, "download": 50.0}
BUDGET = {"public": (300.0, 60.0), "partner": (3000.0, 60.0)}


def endpoint_class(path: str) -> str:
    for name in COST:
        if f"/{name}" in path:
            return name
    return "features"


@app.middleware("http")
async def rate_limit(request: Request, call_next):
    client = redis.Redis(connection_pool=pool)
    key, tier = limit_key(request)
    capacity, window = BUDGET[tier]
    cost = COST[endpoint_class(request.url.path)]

    allowed, remaining = consume(client, f"rl:{key}", capacity,
                                 capacity / window, cost)
    if not allowed:
        wait = int((cost - float(remaining)) / (capacity / window)) + 1
        return JSONResponse(
            {"detail": "rate limit exceeded", "retry_after": wait},
            status_code=429,
            headers={"Retry-After": str(wait),
                     "X-RateLimit-Limit": str(int(capacity)),
                     "X-RateLimit-Remaining": "0"})

    response = await call_next(request)
    if response.status_code == 304:            # refund most of a cheap response
        consume(client, f"rl:{key}", capacity, capacity / window, -cost * 0.9)
    response.headers["X-RateLimit-Limit"] = str(int(capacity))
    response.headers["X-RateLimit-Remaining"] = str(int(float(remaining)))
    return response

Example 2 — protecting the expensive endpoints with a concurrency limit 

import asyncio
from fastapi import HTTPException

HEAVY = asyncio.Semaphore(4)          # at most four bulk exports at once


@app.get("/download")
async def download():
    try:
        await asyncio.wait_for(HEAVY.acquire(), timeout=0.5)
    except asyncio.TimeoutError:
        raise HTTPException(503, "too many exports in progress; try again shortly",
                            headers={"Retry-After": "30"})
    try:
        return await build_export()
    finally:
        HEAVY.release()

A rate limit bounds the arrival rate; a semaphore bounds concurrency. They protect against different failures: the first against a client asking too often, the second against four clients each asking for something that takes thirty seconds.

Measured, the unbounded version of that endpoint served 2.3 requests per second while returning 12.4 MB each — four concurrent copies is most of a server.

Example 3 — testing the limiter 

import time
from fastapi.testclient import TestClient


def test_burst_is_allowed_then_limited(client: TestClient):
    statuses = [client.get("/tiles/v1/10/512/340.mvt").status_code
                for _ in range(400)]
    assert 200 in statuses, "the burst should be allowed"
    assert 429 in statuses, "the limit should eventually apply"


def test_429_carries_retry_after(client: TestClient):
    for _ in range(1000):
        response = client.get("/features?limit=50")
        if response.status_code == 429:
            assert "retry-after" in response.headers
            assert int(response.headers["retry-after"]) > 0
            return
    raise AssertionError("never hit the limit — is the limiter wired up?")


def test_expensive_endpoints_cost_more(client: TestClient):
    tiles = count_until_limited(client, "/tiles/v1/10/512/340.mvt")
    downloads = count_until_limited(client, "/download")
    assert downloads < tiles / 10, ("a download should consume far more budget "
                                    "than a tile")

The third test is the one that encodes the design. Without it, somebody eventually "simplifies" the cost table and the API is back to counting requests.

Explanation

Why request counts are the wrong unit for spatial APIs

The measured spread — 20 kB tiles against 54 MB feature dumps — means a request is not a unit of anything. A limit expressed in requests is implicitly a limit expressed in the cost of the average request, and the average is meaningless when the distribution spans three orders of magnitude.

Cost weighting restores the property you actually want: a caller's budget corresponds to the load they impose.

Why a token bucket suits map traffic

Panning a web map requests a screenful of tiles at once — often twenty or more — and then nothing until the user moves again. A sliding-window limiter sees that burst as abuse; a token bucket sees it as a burst against a capacity, which is exactly what it is.

Set the capacity to at least one screenful and the refill rate to the sustained rate you can afford. The client gets a smooth experience and the server gets a bounded average.

Why the state must be shared

A process-local limiter under eight workers enforces eight independent limits, and which one a request lands on is a load-balancing detail. The effective limit is eight times the intended one, and it is not reproducible.

Redis with an atomic script is the standard answer. The cost is a round trip per request — sub-millisecond on a local instance, and negligible against the tens of milliseconds a spatial request already takes.

Why refunding cheap responses matters

The measured cached path — 1,191 requests per second for a 304 — is exactly the behaviour you want to encourage. Charging it the full cost of a generated response penalises the clients doing the right thing and pushes them towards cache-busting.

Charging on the way out, with a discount for 304s and empty tiles, aligns the incentive with the load. It is a few lines and it changes how well-behaved clients can be.

Table of four rate-limit response headers with examples and reasons.
Sending Remaining on success lets a good client slow itself down first.

Edge cases or notes

  • Process-local state is not a rate limit when there is more than one worker.
  • Trust X-Forwarded-For only behind a proxy you control, and take the first entry.
  • An IP can be a whole office. Anonymous budgets need to accommodate NAT.
  • Retry-After on every 429, or clients amplify the load.
  • Rate limits and concurrency limits are different controls — expensive endpoints need both.
  • Do not rate limit health checks or your own monitoring.
  • Log the caller id on every rejection, or abuse cannot be attributed.
  • Test that expensive endpoints exhaust the budget faster, or the weighting will be removed by a later refactor.

FAQ

Why not just limit requests per minute?

Because spatial requests differ by three orders of magnitude: a tile is about 20 kB and a full feature download was measured at 54 MB. One number cannot be right for both.

Which algorithm should I use?

A token bucket for map traffic — it allows the burst a pan produces and then settles to the average rate. A sliding window punishes exactly the behaviour a map client cannot avoid.

Can I keep the limiter state in memory?

Only with one worker. With eight, each enforces its own limit and the effective rate is eight times the intended one. Use Redis with an atomic script.

Should I limit by IP or by API key?

By key where there is one. Fall back to IP for anonymous callers, remembering that an office behind NAT shares an address.

What must a 429 include?

Retry-After. Without it, clients with retry loops turn one rejection into many, which is the traffic pattern the limit exists to stop.

Should a 304 cost the same as a full response?

No. Measured, the cached path ran at 1,191 requests per second against 100 for generating the body. Charge on the way out and discount the cheap responses.