How to Query the Overpass API from Python
Problem statement
OSMnx is a good wrapper, and wrappers have edges. Sooner or later you want something it does not express:
- every feature with
amenity=cafeandoutdoor_seating=yes— an intersection, not a union - a count, without downloading 30,000 features to call
len()on them - features inside an administrative area named in the data, not a bounding box
- everything that changed in the last thirty days
All of these are one line of Overpass QL. The cost of dropping to the raw API is that you now own the failure handling, and Overpass fails in ways that do not look like failures:
import requests
r = requests.post("https://overpass-api.de/api/interpreter", data={"data": query})
data = r.json()requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
That is not a malformed response. That is Overpass returning an HTML error page with HTTP 504, because the public server was busy — and r.json() never gets far enough to tell you so.
Quick answer
POST the query, check the content before parsing it, and read the error out of the HTML:
import re
import requests
OVERPASS = "https://overpass-api.de/api/interpreter"
HEADERS = {"User-Agent": "my-project/1.0 ([email protected])"}
def overpass(query, timeout=180):
r = requests.post(OVERPASS, data={"data": query}, headers=HEADERS, timeout=timeout)
if r.status_code != 200 or not r.text.lstrip().startswith("{"):
errors = [" ".join(re.sub("<[^>]+>", "", e).split())
for e in re.findall(r"<p><strong.*?</p>", r.text, re.S)]
raise RuntimeError(f"Overpass HTTP {r.status_code}: {errors or r.text[:200]}")
return r.json()
data = overpass("""
[out:json][timeout:60];
node["amenity"="cafe"](53.4770,-2.2500,53.4850,-2.2350);
out body;
""")
print(len(data["elements"]), "elements")87 elements
Note the bounding box order: (south, west, north, east) — latitude first. That is the opposite of OSMnx and of GeoPandas total_bounds.
Step-by-step solution
1. Learn the four parts of a query
[out:json][timeout:60]; ← settings: response format and server-side time budget
node["amenity"="cafe"] ← element type + tag filters
(53.477,-2.250,53.485,-2.235); ← spatial constraint
out body; ← what to return
Every statement ends in a semicolon, including the settings block and the final out. A missing semicolon is the single most common cause of an HTTP 400 with an HTML body.
2. Combine element types with a union
A tag can be on a node, a way or a relation. Cafés are usually nodes; parks are usually ways. If you do not know, ask for all three:
[out:json][timeout:60];
(
node["leisure"="park"](53.44,-2.30,53.52,-2.18);
way["leisure"="park"](53.44,-2.30,53.52,-2.18);
relation["leisure"="park"](53.44,-2.30,53.52,-2.18);
);
out center;
The parentheses make a union. out center; returns each way and relation with a computed centroid instead of its full geometry:
element = data["elements"][0]
print(list(element.keys()))
print(element["center"])['type', 'id', 'center', 'nodes', 'tags']
{'lat': 53.4553571, 'lon': -2.2132332}
That is a fraction of the bytes of full geometry, and it is all you need for a point analysis. When you do need the shapes, use out geom; instead.
3. Use tag filters to express AND
This is the thing OSMnx will not do for you. Stack filters on one element and they combine with AND:
node["amenity"="cafe"]["outdoor_seating"="yes"](53.44,-2.30,53.52,-2.18);
The full filter vocabulary:
| Filter | Meaning |
|---|---|
["amenity"] |
key exists, any value |
["amenity"="cafe"] |
exact value |
["amenity"!="cafe"] |
not this value (includes elements with no such key) |
["name"~"Coffee"] |
value matches a regex |
["name"~"coffee",i] |
case-insensitive regex |
[!"name"] |
key absent |
4. Ask for a count before you ask for the data
[out:json][timeout:60];
node["amenity"="cafe"](53.44,-2.30,53.52,-2.18);
out count;
print(data["elements"])[{'type': 'count', 'id': 0,
'tags': {'nodes': '297', 'ways': '0', 'relations': '0', 'total': '297'}}]
One tiny response instead of a megabyte. Two things to notice: the counts are strings, so int() them; and the split by element type tells you in advance whether you need out center for ways.
5. Query by area, not just by box
An area is derived from a closed way or relation that Overpass has indexed. Bind it to a variable and use it as a constraint:
[out:json][timeout:60];
area["name"="Manchester"]["admin_level"="8"]->.searchArea;
node["amenity"="cafe"](area.searchArea);
out body;
->.searchArea names the result; (area.searchArea) constrains to it. This is exact — the actual administrative boundary rather than a box around it.
Qualify the area filter. area["name"="Ancoats"] alone matches any object named Ancoats anywhere on earth and forces a global scan, which is a reliable way to earn a 504. Always add admin_level, boundary, or an ISO3166 code.
Code examples
Example 1 — a client that handles the real failure modes
import re
import time
import requests
OVERPASS = "https://overpass-api.de/api/interpreter"
HEADERS = {"User-Agent": "spatialworkflow-example/1.0 ([email protected])"}
RETRYABLE = {429, 502, 503, 504}
def _errors(html):
"""Overpass reports errors as <p><strong>Error</strong>: …</p> inside an HTML page."""
found = re.findall(r"<p><strong.*?</p>", html, re.S)
return [" ".join(re.sub("<[^>]+>", "", block).split()) for block in found]
def overpass(query, *, attempts=4, timeout=300):
delay = 5
for attempt in range(1, attempts + 1):
r = requests.post(OVERPASS, data={"data": query}, headers=HEADERS, timeout=timeout)
if r.status_code == 200 and r.text.lstrip().startswith("{"):
payload = r.json()
# A remark means the server gave up mid-query and sent partial data.
if "remark" in payload:
raise RuntimeError(f"Overpass partial result: {payload['remark']}")
return payload
messages = _errors(r.text) or [r.text[:160]]
if r.status_code in RETRYABLE and attempt < attempts:
print(f" attempt {attempt}: HTTP {r.status_code} — retrying in {delay}s")
time.sleep(delay)
delay *= 2
continue
raise RuntimeError(f"Overpass HTTP {r.status_code}: {messages[0]}")
raise RuntimeError("unreachable")
data = overpass("""
[out:json][timeout:120];
node["amenity"="cafe"](53.4770,-2.2500,53.4850,-2.2350);
out body;
""")
print(len(data["elements"]), "cafes") attempt 1: HTTP 504 — retrying in 5s
87 cafes
Three things this handles that the naive version does not: a busy server (504, retryable), a rate limit (429, retryable), and the remark key — which arrives with HTTP 200 and valid JSON when the server times out mid-query and returns whatever it had. That last one is the dangerous case, because it looks exactly like success with fewer results.
Example 2 — turning the response into a GeoDataFrame
import geopandas as gpd
import pandas as pd
from shapely.geometry import Point, LineString, Polygon
def to_gdf(payload, crs="EPSG:4326"):
rows, geoms = [], []
for element in payload["elements"]:
if element["type"] == "node":
geom = Point(element["lon"], element["lat"])
elif "center" in element:
geom = Point(element["center"]["lon"], element["center"]["lat"])
elif "geometry" in element: # from `out geom;`
coords = [(p["lon"], p["lat"]) for p in element["geometry"]]
closed = len(coords) > 3 and coords[0] == coords[-1]
geom = Polygon(coords) if closed else LineString(coords)
else:
continue # no usable geometry
rows.append({"element": element["type"], "id": element["id"], **element.get("tags", {})})
geoms.append(geom)
return gpd.GeoDataFrame(pd.DataFrame(rows), geometry=geoms, crs=crs)
cafes = to_gdf(data)
print(cafes.shape)
print(cafes[["element", "id", "name", "cuisine"]].head(3).to_string(index=False))(87, 41)
element id name cuisine
node 249517666 Katsouris Deli NaN
node 324661012 Pret A Manger coffee_shop
node 324661296 Caffe Nero coffee_shop
Whether a closed way is a polygon or a line is a judgement — see the closed-way ambiguity. The rule above (closed and more than three points) is right for buildings and landuse, and wrong for roundabouts. If you are mixing feature types, split the query instead of the parsing.
Example 3 — the intersection query OSMnx cannot express
BBOX = "53.44,-2.30,53.52,-2.18"
QUERIES = {
"all cafes": f'node["amenity"="cafe"]({BBOX});',
"with outdoor seating": f'node["amenity"="cafe"]["outdoor_seating"="yes"]({BBOX});',
"wheelchair accessible": f'node["amenity"="cafe"]["wheelchair"="yes"]({BBOX});',
"both": f'node["amenity"="cafe"]["outdoor_seating"="yes"]["wheelchair"="yes"]({BBOX});',
"unnamed": f'node["amenity"="cafe"][!"name"]({BBOX});',
}
for label, selector in QUERIES.items():
payload = overpass(f"[out:json][timeout:60];\n{selector}\nout count;")
total = int(payload["elements"][0]["tags"]["total"])
print(f"{label:24} {total:5}")all cafes 297
with outdoor seating 34
wheelchair accessible 61
both 12
unnamed 18
Five real questions, five tiny responses, no feature data downloaded at all. Stacked filters give AND; [!"name"] gives absence. Neither is expressible through the tags dict of a wrapper.
Explanation
Why r.json() is the wrong first move
Overpass signals almost everything through the body rather than the status line. A syntax error is HTTP 400 with an HTML page. A busy dispatcher is HTTP 504 with an HTML page. A rate limit is HTTP 429 with an HTML page. Only a successful query is JSON.
So r.json() raises JSONDecodeError for four unrelated problems, and the traceback names none of them. Checking r.text.lstrip().startswith("{") before parsing, and scraping the <p><strong>Error</strong> block when it does not, turns all four into messages you can act on:
Overpass HTTP 504: Error: runtime error: open64: 0 Success /osm3s_osm_base
Dispatcher_Client::request_read_and_idx::timeout. The server is probably too busy
to handle your request.
Why remark is the one to fear
The [timeout:60] in the settings block is a server-side budget. When a query exceeds it, Overpass does not fail — it returns HTTP 200, valid JSON, the elements it managed to collect, and a top-level remark explaining that the query ran out of time.
Code that checks only the status code accepts this as a complete answer. A weekly job silently returning 60% of the cafés is far worse than one that crashes. Always check for remark, and treat it as an error.
Why the two timeouts are different
There are two, and they are unrelated:
[timeout:180]inside the query — how long the Overpass server will spend before giving up and sending aremark.requests.post(..., timeout=300)— how long your client waits for bytes.
The client timeout must be the larger of the two, or you will disconnect from queries the server would have completed. Setting the server timeout high does not make a query faster; it makes a slow query finish rather than truncate.
Why to be polite to the public endpoint
overpass-api.de is donated infrastructure with a shared quota. The etiquette that keeps you unblocked:
- Send a real
User-Agentwith a contact address. - Ask for
out countbeforeout bodyon anything you have not sized. - Cache results locally, so iterating on your analysis does not re-query.
- Use exponential backoff on 429 and 504 — never a tight retry loop.
- For anything scheduled or bulk, move to a regional extract or your own instance.
Edge cases or notes
- Bounding box order is
(south, west, north, east)— latitude first, the opposite of OSMnx.total_boundsfrom GeoPandas gives(minx, miny, maxx, maxy), so reorder it explicitly. out bodyon ways returns node references, not coordinates. Useout geom;for full geometry orout center;for a centroid.["amenity"!="cafe"]also matches elements with noamenitykey at all. Negation in Overpass is not the complement of a set you can enumerate.- Areas exist only for objects Overpass has indexed as areas — closed ways and multipolygon relations with certain tags. A
place=neighbourhoodnode has no area, soarea["name"="Ancoats"]finds nothing. - Regex filters are slow.
["name"~"Coffee"]on a large extent will time out where an exact match would not. [date:"2026-01-01T00:00:00Z"]in the settings block queries a historic snapshot, and[diff:...]returns what changed — both are much heavier than a current query.- Different mirrors have different limits.
overpass.kumi.systemsand others exist; swapping the base URL is a one-line change and worth having configurable.
Internal links
- The OpenStreetMap data model explained — what nodes, ways, relations and tags actually are
- How to download OpenStreetMap data in Python with OSMnx — the wrapper, and when it is enough
- OSMnx download fails, hangs or times out — the same server problems seen through the wrapper
- How to cache downloaded GIS data so you fetch it once — stop re-querying while you iterate
- How to retry a flaky GIS step in Python — the general backoff pattern used above
- GIS data sources explained — ODbL, attribution and what you may publish
- GeoJSON downloaded from an API is empty or truncated — the same partial-result trap in other services
- How to read a CSV with coordinates as a GeoDataFrame — building geometry from raw coordinate pairs
FAQ
Why does r.json() raise JSONDecodeError?
Because the response is an HTML error page, not JSON. Overpass reports syntax errors, rate limits and server load with HTML bodies. Check that the body starts with { before parsing, and scrape the error text when it does not.
What is the remark key?
A server-side timeout. You get HTTP 200, valid JSON and a partial element list. Treat any response containing remark as a failure, or you will silently analyse a fraction of the data.
How do I query "A and B" rather than "A or B"?
Stack tag filters on one element selector: node["amenity"="cafe"]["wheelchair"="yes"]. Separate statements inside ( … ); are a union, which is OR.
Why is my area query so slow?
The area filter is probably unqualified. area["name"="X"] scans globally. Add ["admin_level"="8"] or ["boundary"="administrative"] to hit the index.
What is the difference between out body, out center and out geom?
body returns tags plus node references (no coordinates for ways). center adds a computed centroid. geom inlines the full coordinate list. Use center for point analysis and geom when you need the shape.
How many requests can I make?
There is no published number — the public endpoint enforces slots dynamically and returns 429 when you exceed them. Back off exponentially, cache aggressively, and move to your own instance for scheduled work.
Can I query historic OSM data?
Yes, with [date:"…"] in the settings block for a snapshot or [diff:"…","…"] for changes. Both are far more expensive than a current query, so size them with out count first.