How to Rate Limit and Throttle a Spatial API
On this page
Problem statement
Rate limiting a spatial API is not the same problem as rate limiting a REST API, because the requests are not comparable. Measured on the services built elsewhere on this site — a vector tile of central Leiden and a feature API over Natural Earth's provinces:
request payload one client, back to back
one zoom-15 vector tile 69.6 kB
50-feature page 859.3 kB 47.6 req/s
200-feature page 3,380.2 kB 14.2 req/s
whole layer 54.29 MB
A limit of "100 requests per minute" either strangles a map that legitimately needs a screenful of tiles at a time, or permits a hundred 54 MB downloads. The two are three orders of magnitude apart, and no single request count is right for both.
The second complication is operational: a limiter that keeps its state in a process is not a limiter when there are eight workers.
Quick answer
Weight by cost, share the state, and always send Retry-After:
import time
from fastapi import HTTPException
COST = {"tile": 1.0, "features": 5.0, "download": 50.0}
BUDGETS = {"public": (300, 60), "partner": (3000, 60)} # units, per seconds
class TokenBucket:
def __init__(self, store):
self.store = store # Redis in production
def consume(self, key, capacity, per_seconds, cost=1.0):
rate = capacity / per_seconds
tokens, updated = self.store.get(key, (capacity, time.monotonic()))
now = time.monotonic()
tokens = min(capacity, tokens + (now - updated) * rate)
if tokens < cost:
wait = (cost - tokens) / rate
raise HTTPException(429, "rate limit exceeded", headers={
"Retry-After": str(int(wait) + 1),
"X-RateLimit-Limit": str(int(capacity)),
"X-RateLimit-Remaining": "0",
"X-RateLimit-Reset": str(int(time.time() + wait))})
self.store.set(key, (tokens - cost, now))
return int(tokens - cost)class DictStore(dict):
"""The quick answer's store, in memory — fine for one process only."""
def set(self, key, value):
self[key] = value
bucket = TokenBucket(DictStore())
for kind in ("tile", "features", "download"):
served = 0
try:
while True:
bucket.consume(f"ip:198.51.100.7:{kind}", *BUDGETS["public"], cost=COST[kind])
served += 1
except HTTPException as exc:
print(f"{kind:9} {served:4} served in the burst, then {exc.status_code}, "
f"Retry-After {exc.headers['Retry-After']} s")tile 300 served in the burst, then 429, Retry-After 1 s
features 60 served in the burst, then 429, Retry-After 1 s
download 6 served in the burst, then 429, Retry-After 10 s
A token bucket allows a burst up to the capacity and then settles to the average rate, which matches how map clients behave: a pan requests twenty tiles at once and then nothing for ten seconds.
Price endpoints, pick an algorithm, share the state
1. Price the endpoints
Assign each endpoint a cost roughly proportional to the work it causes. Approximate is fine; the point is to stop treating a tile and a bulk download as equivalent.
COST = {
"GET /tiles/{z}/{x}/{y}": 1.0,
"GET /features": 5.0,
"GET /features/stream": 20.0,
"GET /download": 50.0,
}A useful calibration: make one unit approximately the cheapest request, then scale from measured payloads and timings. The measured ratio between a 69.6 kB tile and a 3.4 MB page justifies a factor of about fifty.
2. Choose the algorithm from the traffic shape
- Token bucket — allows bursts up to a capacity, then the average rate. Right for map clients, which burst by nature.
- Sliding window — a strict count over the last N seconds. Fairer, and it punishes the burst that a map tile load legitimately produces.
- Leaky bucket — a constant drain rate. Right for protecting a fragile downstream service.
For a tile endpoint the token bucket is nearly always the right choice, with a capacity large enough for one screenful of tiles.
3. Share the state, or the limit is a lie
Eight uvicorn workers with a process-local dictionary enforce eight times the intended limit, distributed unpredictably. Use Redis:
import redis
LUA = """
local tokens_key = KEYS[1]
local capacity = tonumber(ARGV[1])
local rate = tonumber(ARGV[2])
local now = tonumber(ARGV[3])
local cost = tonumber(ARGV[4])
local data = redis.call('HMGET', tokens_key, 'tokens', 'updated')
local tokens = tonumber(data[1]) or capacity
local updated = tonumber(data[2]) or now
tokens = math.min(capacity, tokens + (now - updated) * rate)
if tokens < cost then
return {0, tostring(tokens)}
end
tokens = tokens - cost
redis.call('HMSET', tokens_key, 'tokens', tokens, 'updated', now)
redis.call('EXPIRE', tokens_key, 3600)
return {1, tostring(tokens)}
"""def consume(client, key, capacity, rate, cost):
"""One atomic token-bucket step in Redis: (allowed, tokens left)."""
allowed, remaining = client.register_script(LUA)(
keys=[key], args=[capacity, rate, time.time(), cost])
return bool(allowed), float(remaining)
client = redis.Redis.from_url(REDIS_URL)
client.delete("rl:demo")
started = time.perf_counter()
results = [consume(client, "rl:demo", 300, 5.0, 1.0)[0] for _ in range(320)]
per_call = (time.perf_counter() - started) / 320 * 1000
print(f"{sum(results)} of 320 allowed; the 301st was {'allowed' if results[300] else 'refused'}; "
f"{per_call:.2f} ms per check")300 of 320 allowed; the 301st was refused; 0.07 ms per check
The Lua script makes the read-modify-write atomic, which a Python round trip does not.
4. Key by caller, with a sensible fallback
import hashlib
def hash_key(key: str) -> str:
"""Never store the raw API key, even as a rate-limit key."""
return hashlib.sha256(key.encode()).hexdigest()[:16]
def limit_key(request) -> tuple[str, str]:
if (key := request.headers.get("x-api-key")):
return f"key:{hash_key(key)}", "partner"
forwarded = request.headers.get("x-forwarded-for", "")
ip = forwarded.split(",")[0].strip() or request.client.host
return f"ip:{ip}", "public"Trust X-Forwarded-For only behind a proxy you control, and take the first entry. Remember that an IP can be an entire office, so the anonymous budget should be generous enough for a shared address and small enough to bound abuse.
5. Send the headers that let clients behave
429 Too Many Requests
Retry-After: 12
X-RateLimit-Limit: 300
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 1757068812Retry-After is the one that matters. Without it a client with a retry loop turns one rejection into a hundred, which is the exact traffic pattern the limit exists to prevent.
Sending X-RateLimit-Remaining on successful responses lets a well-behaved client slow itself down before being rejected.
6. Exempt the cheap paths
A 304 response costs almost nothing — measured in Example 1, 2,884 requests per second against 1,329 for sending the 857 kB body. Charging it the same as a full response penalises exactly the clients that are caching properly.
Charge on the way out, based on what was actually done:
cost = 0.1 if response.status_code == 304 else COST[endpoint]Cost-weighted limits backed by Redis
The data used on this page
The limiter's state lives in Redis, run here as a throwaway local container. The feature endpoint serves the first 50 of Natural Earth's 1:10m states and provinces — the 859 kB page measured on the filtering page — so that a 200 and a 304 can be compared on a real payload.
import io
import os
import subprocess
import time
import zipfile
import geopandas as gpd
import redis
import requests
if subprocess.run(["docker", "inspect", "guide-redis"], capture_output=True).returncode != 0:
subprocess.run(["docker", "run", "-d", "--name", "guide-redis", "-p", "127.0.0.1:56379:6379",
"redis:8-alpine"], check=True, capture_output=True)
REDIS_URL = "redis://127.0.0.1:56379/0"
for _ in range(100):
try:
redis.Redis.from_url(REDIS_URL).ping()
break
except redis.ConnectionError:
time.sleep(0.2)
if not os.path.exists("page.geojson"):
archive = requests.get("https://naciscdn.org/naturalearth/10m/cultural/"
"ne_10m_admin_1_states_provinces.zip", timeout=300)
with zipfile.ZipFile(io.BytesIO(archive.content)) as z:
for ext in (".shp", ".shx", ".dbf", ".prj", ".cpg"):
open(f"ne_provinces{ext}", "wb").write(z.read(f"ne_10m_admin_1_states_provinces{ext}"))
page = gpd.read_file("ne_provinces.shp", rows=50)[["name", "admin", "geometry"]]
open("page.geojson", "w").write(page.to_json())
print("redis", redis.Redis.from_url(REDIS_URL).info()["redis_version"],
f"| page.geojson {os.path.getsize('page.geojson') / 1e3:.1f} kB")redis 8.10.1 | page.geojson 857.1 kB
Example 1 — middleware with cost weighting and Redis state
The service is a file so that it can run under uvicorn with several workers. LIMITER switches the state between Redis, a process-local dictionary and nothing, which is what lets Example 1 measure the difference.
%%writefile limited_app.py
import asyncio
import hashlib
import json
import os
import time
import redis
from fastapi import FastAPI, HTTPException, Request, Response
from fastapi.responses import JSONResponse
app = FastAPI()
pool = redis.ConnectionPool.from_url(os.environ.get("REDIS_URL", "redis://127.0.0.1:56379/0"))
LIMITER = os.environ.get("LIMITER", "redis") # redis | local | off
COST = {"tiles": 1.0, "features": 5.0, "stream": 20.0, "download": 50.0}
BUDGET = {"public": (300.0, 60.0), "partner": (3000.0, 60.0)}
LUA = """
local data = redis.call('HMGET', KEYS[1], 'tokens', 'updated')
local capacity, rate = tonumber(ARGV[1]), tonumber(ARGV[2])
local now, cost = tonumber(ARGV[3]), tonumber(ARGV[4])
local tokens = tonumber(data[1]) or capacity
local updated = tonumber(data[2]) or now
tokens = math.min(capacity, tokens + (now - updated) * rate)
if tokens < cost then return {0, tostring(tokens)} end
tokens = tokens - cost
redis.call('HMSET', KEYS[1], 'tokens', tokens, 'updated', now)
redis.call('EXPIRE', KEYS[1], 3600)
return {1, tostring(tokens)}
"""
LOCAL = {} # per process: the thing not to do
BODY = open("page.geojson").read()
ETAG = '"' + hashlib.sha256(BODY.encode()).hexdigest()[:16] + '"'
def consume(key, capacity, rate, cost):
if LIMITER == "redis":
client = redis.Redis(connection_pool=pool)
allowed, remaining = client.register_script(LUA)(keys=[key], args=[capacity, rate, time.time(), cost])
return bool(allowed), float(remaining)
tokens, updated = LOCAL.get(key, (capacity, time.monotonic()))
now = time.monotonic()
tokens = min(capacity, tokens + (now - updated) * rate)
if tokens < cost:
return False, tokens
LOCAL[key] = (tokens - cost, now)
return True, tokens - cost
def limit_key(request):
if (key := request.headers.get("x-api-key")):
return "key:" + hashlib.sha256(key.encode()).hexdigest()[:16], "partner"
return "ip:" + request.client.host, "public"
def endpoint_class(path: str) -> str:
for name in COST:
if f"/{name}" in path:
return name
return "features"
@app.middleware("http")
async def rate_limit(request: Request, call_next):
if LIMITER == "off" or request.url.path == "/health":
return await call_next(request)
key, tier = limit_key(request)
capacity, window = BUDGET[tier]
cost = COST[endpoint_class(request.url.path)]
allowed, remaining = consume(f"rl:{key}", capacity, capacity / window, cost)
if not allowed:
wait = int((cost - remaining) / (capacity / window)) + 1
return JSONResponse({"detail": "rate limit exceeded", "retry_after": wait}, status_code=429,
headers={"Retry-After": str(wait), "X-RateLimit-Limit": str(int(capacity)),
"X-RateLimit-Remaining": "0"})
response = await call_next(request)
if response.status_code == 304: # refund most of a cheap response
allowed, remaining = consume(f"rl:{key}", capacity, capacity / window, -cost * 0.9)
response.headers["X-RateLimit-Limit"] = str(int(capacity))
response.headers["X-RateLimit-Remaining"] = str(int(remaining))
return response
@app.get("/health")
def health():
return {"ok": True}
@app.get("/tiles/{z}/{x}/{y}.mvt")
def tile(z: int, x: int, y: int):
return Response(b"\x1a\x00", media_type="application/vnd.mapbox-vector-tile")
@app.get("/features")
def features(request: Request):
if request.headers.get("if-none-match") == ETAG:
return Response(status_code=304, headers={"ETag": ETAG})
return Response(BODY, media_type="application/geo+json", headers={"ETag": ETAG})Two things measured against it. First, how many tile requests a four-worker server lets through a single client before refusing, with the state in Redis and with the process-local dictionary; second, what a 304 costs against the full body.
import socket
import sys
import httpx
def start(env, workers=1):
with socket.socket() as probe:
probe.bind(("127.0.0.1", 0))
port = probe.getsockname()[1]
proc = subprocess.Popen([sys.executable, "-m", "uvicorn", "limited_app:app", "--port", str(port),
"--workers", str(workers), "--log-level", "warning"],
env={**os.environ, **env}, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
base = f"http://127.0.0.1:{port}"
for _ in range(200):
try:
httpx.get(base + "/health", timeout=1)
return proc, base
except httpx.TransportError:
time.sleep(0.1)
raise RuntimeError("server did not start")
for limiter in ("redis", "local"):
redis.Redis.from_url(REDIS_URL).flushdb()
proc, base = start({"LIMITER": limiter}, workers=4)
statuses = [httpx.get(f"{base}/tiles/15/16793/10800.mvt").status_code for _ in range(2000)]
proc.terminate(); proc.wait()
print(f"{limiter:6} limiter, 4 workers: {statuses.count(200):5,} of 2,000 tile requests served "
f"before and between refusals (budget 300)")
proc, base = start({"LIMITER": "off"})
for label, headers in (("200 with the body", {}), ("304 not modified", {"If-None-Match": httpx.get(base + "/features").headers["etag"]})):
with httpx.Client() as session:
n, started = 0, time.perf_counter()
while time.perf_counter() - started < 3:
response = session.get(base + "/features", headers=headers)
n += 1
print(f"{label:18} {n / (time.perf_counter() - started):7.1f} req/s, {len(response.content):7,} bytes")
proc.terminate(); proc.wait()redis limiter, 4 workers: 335 of 2,000 tile requests served before and between refusals (budget 300)
local limiter, 4 workers: 1,335 of 2,000 tile requests served before and between refusals (budget 300)
200 with the body 1328.8 req/s, 857,061 bytes
304 not modified 2883.8 req/s, 0 bytes
The Redis limiter held the budget across four workers: 335 is the 300 budget plus what refilled at five tokens a second while the test ran. The dictionary gave each worker its own budget and let through 1,335 — about four budgets' worth, split by whichever worker the kernel handed each connection to.
Example 2 — protecting the expensive endpoints with a concurrency limit
%%writefile -a limited_app.py
import asyncio
from fastapi import HTTPException
HEAVY = asyncio.Semaphore(4) # at most four bulk exports at once
async def build_export():
await asyncio.sleep(3) # stands in for thirty seconds of real work
return {"export": "ready"}
@app.get("/download")
async def download():
try:
await asyncio.wait_for(HEAVY.acquire(), timeout=0.5)
except asyncio.TimeoutError:
raise HTTPException(503, "too many exports in progress; try again shortly",
headers={"Retry-After": "30"})
try:
return await build_export()
finally:
HEAVY.release()import asyncio
async def six_at_once(base):
async with httpx.AsyncClient(timeout=None) as session:
responses = await asyncio.gather(*(session.get(base + "/download",
headers={"x-api-key": f"client-{i}"})
for i in range(6)))
return [r.status_code for r in responses]
proc, base = start({"LIMITER": "redis"})
redis.Redis.from_url(REDIS_URL).flushdb()
print(sorted(asyncio.run(six_at_once(base))))
proc.terminate(); proc.wait()[200, 200, 200, 200, 503, 503]
Six different partners, each well inside their rate budget, asked at once: four got an export slot, and two got a 503 with Retry-After instead of a server with six exports competing for it.
A rate limit bounds the arrival rate; a semaphore bounds concurrency. They protect against different failures: the first against a client asking too often, the second against four clients each asking for something that takes thirty seconds.
Measured on the filtering page, the largest allowed feature page served 14.2 requests per second at 3.4 MB each to one client; a bulk export of the whole layer is sixteen times that payload.
Example 3 — testing the limiter
%%writefile test_limits.py
import os
import pytest
import redis
from fastapi.testclient import TestClient
os.environ["LIMITER"] = "redis"
import limited_app # noqa: E402
@pytest.fixture
def client():
redis.Redis.from_url("redis://127.0.0.1:56379/0").flushdb()
return TestClient(limited_app.app)
def count_until_limited(client, path, attempts=2000):
for n in range(attempts):
if client.get(path).status_code == 429:
return n
return attempts
def test_burst_is_allowed_then_limited(client: TestClient):
statuses = [client.get("/tiles/10/512/340.mvt").status_code
for _ in range(400)]
assert 200 in statuses, "the burst should be allowed"
assert 429 in statuses, "the limit should eventually apply"
def test_429_carries_retry_after(client: TestClient):
for _ in range(1000):
response = client.get("/features?limit=50")
if response.status_code == 429:
assert "retry-after" in response.headers
assert int(response.headers["retry-after"]) > 0
return
raise AssertionError("never hit the limit — is the limiter wired up?")
def test_expensive_endpoints_cost_more(client: TestClient):
tiles = count_until_limited(client, "/tiles/10/512/340.mvt")
redis.Redis.from_url("redis://127.0.0.1:56379/0").flushdb() # same caller: start the count clean
downloads = count_until_limited(client, "/download")
assert downloads < tiles / 10, ("a download should consume far more budget "
"than a tile")!{sys.executable} -m pytest -q test_limits.py 2>&1 | tail -3... [100%]
3 passed in 25.18s
The third test is the one that encodes the design. Without it, somebody eventually "simplifies" the cost table and the API is back to counting requests.
Why request counts are the wrong unit
Why request counts are the wrong unit for spatial APIs
The measured spread — a 69.6 kB tile against a 54 MB layer — means a request is not a unit of anything. A limit expressed in requests is implicitly a limit expressed in the cost of the average request, and the average is meaningless when the distribution spans three orders of magnitude.
Cost weighting restores the property you actually want: a caller's budget corresponds to the load they impose.
Why a token bucket suits map traffic
Panning a web map requests a screenful of tiles at once — often twenty or more — and then nothing until the user moves again. A sliding-window limiter sees that burst as abuse; a token bucket sees it as a burst against a capacity, which is exactly what it is.
Set the capacity to at least one screenful and the refill rate to the sustained rate you can afford. The client gets a smooth experience and the server gets a bounded average.
Why the state must be shared
A process-local limiter under eight workers enforces eight independent limits, and which one a request lands on is a load-balancing detail. The effective limit is eight times the intended one, and it is not reproducible.
Redis with an atomic script is the standard answer. The cost is a round trip per request — 0.07 ms per check against the local instance in step 3, negligible beside the work a spatial request already does.
Why refunding cheap responses matters
The measured cached path — 2,884 requests per second for a 304, against 1,329 for the full 857 kB body — is exactly the behaviour you want to encourage. Charging it the full cost of a generated response penalises the clients doing the right thing and pushes them towards cache-busting.
Charging on the way out, with a discount for 304s and empty tiles, aligns the incentive with the load. It is a few lines and it changes how well-behaved clients can be.
Process-local limits, proxies and offices behind one IP
- Process-local state is not a rate limit when there is more than one worker.
- Trust
X-Forwarded-Foronly behind a proxy you control, and take the first entry. - An IP can be a whole office. Anonymous budgets need to accommodate NAT.
Retry-Afteron every 429, or clients amplify the load.- Rate limits and concurrency limits are different controls — expensive endpoints need both.
- Do not rate limit health checks or your own monitoring.
- Log the caller id on every rejection, or abuse cannot be attributed.
- Test that expensive endpoints exhaust the budget faster, or the weighting will be removed by a later refactor.
Internal links
- Authentication and rate limits for a spatial API — identity and the wider policy
- Serving spatial data explained: files, features and tiles — why the requests differ so much
- Fixing a spatial API that is slow under load — what happens without limits
- HTTP caching for spatial data: ETags, max-age and invalidation — the cheap path worth rewarding
- How to add bounding box and attribute filters to a spatial API — bounding a single request
- Fixing 429 too many requests when geocoding — being on the receiving end
- How to retry flaky steps in a GIS pipeline — backoff on the client side
- How to test a spatial API with pytest and httpx — testing the limiter
FAQ
Why not just limit requests per minute?
Because spatial requests differ by three orders of magnitude: a dense vector tile was 69.6 kB and a whole feature layer 54.29 MB. One number cannot be right for both.
Which algorithm should I use?
A token bucket for map traffic — it allows the burst a pan produces and then settles to the average rate. A sliding window punishes exactly the behaviour a map client cannot avoid.
Can I keep the limiter state in memory?
Only with one worker. Measured with four, the process-local limiter let through 1,335 of 2,000 requests against a budget of 300; the Redis one let through 335. Use Redis with an atomic script.
Should I limit by IP or by API key?
By key where there is one. Fall back to IP for anonymous callers, remembering that an office behind NAT shares an address.
What must a 429 include?
Retry-After. Without it, clients with retry loops turn one rejection into many, which is the traffic pattern the limit exists to stop.
Should a 304 cost the same as a full response?
No. Measured, a 304 served 2,884 requests per second against 1,329 for the full body. Charge on the way out and discount the cheap responses.