🔧 Fix Data sources & downloads Files & formats Finding & Fetching Data · step 9 of 14 Updated

GeoJSON Downloaded from an API Is Empty or Truncated

Problem statement

You request a layer from a service and the number is wrong. Not obviously wrong — plausibly wrong:

import requests

r = requests.get("https://demo.pygeoapi.io/master/collections/lakes/items",
                 params={"f": "json", "limit": 100000}, timeout=60)
data = r.json()
print(r.status_code, len(data["features"]))
200 10

You asked for a hundred thousand. You got ten. The status is 200. The GeoJSON is valid. Nothing raised.

Or the opposite — you get nothing at all:

print(len(gpd.read_file(wfs_url)))
0

Both are the same underlying problem: a service applied a limit or a filter you did not know about, and reported success. The difference between a truncated download and a complete one is not visible in the file. It is only visible if you asked the server how many features there should have been.

Quick answer

Never trust a feature count you did not compare against the server's own:

r = requests.get("https://demo.pygeoapi.io/master/collections/lakes/items",
                 params={"f": "json", "limit": 100000}, timeout=60)
payload = r.json()

matched = payload.get("numberMatched")
returned = payload.get("numberReturned", len(payload["features"]))
print(f"returned {returned} of {matched}")
if matched is not None and returned < matched:
    raise RuntimeError(f"truncated: {returned} of {matched} — page through the result")
returned 10 of 25
RuntimeError: truncated: 10 of 25 — page through the result

The four causes, and how to tell them apart:

Symptom Cause Check
exactly 10, 100, 1000, 5000 features server-side cap numberMatched or resultType=hits
0 features, HTTP 200 bbox in the wrong CRS or units print the bbox and the layer's CRS
0 features, HTTP 200, XML body an OGC exception report check whether the body starts with <
a plausible but short count mid-query timeout look for remark in the JSON
Four ways a spatial API returns an incomplete result while reporting HTTP 200, and the check that identifies each.
All four are HTTP 200. Only the check in the right-hand column tells them apart.

Step-by-step solution

1. Print what the server said, not just what you got

The first move is always the same: look at the response envelope rather than the features.

for key in ("numberMatched", "numberReturned", "remark", "type"):
    if key in payload:
        print(f"{key:16} {payload[key]}")
print(f"{'features':16} {len(payload['features'])}")
print(f"{'links':16} {[l['rel'] for l in payload.get('links', [])]}")
numberMatched    25
numberReturned   10
type             FeatureCollection
features         10
links            ['self', 'alternate', 'alternate', 'alternate', 'next', 'collection']

numberMatched 25, numberReturned 10, and a next link. The server told you plainly that there was more; the code just did not read it.

2. If there is no numberMatched, ask for the count separately

WFS servers frequently omit it from GeoJSON output. Use the dedicated hit-count request:

import re

r = requests.get(BASE, params={
    "service": "WFS", "version": "2.0.0", "request": "GetFeature",
    "typeNames": "wijkenbuurten:buurten", "resultType": "hits",
}, timeout=120)
print(int(re.search(r'numberMatched="(\d+)"', r.text).group(1)))
14515

It returns XML even when you asked for JSON, because the count lives in an attribute of the WFS response element. Parse it with a regex.

3. Recognise the round numbers

A count of exactly 10, 100, 500, 1000, 2000 or 5000 is almost never a coincidence. Those are default caps.

The trap is that asking for more does not raise:

for limit in (10, 1000, 100000):
    n = len(requests.get(URL, params={"f": "json", "limit": limit}, timeout=60).json()["features"])
    print(f"asked {limit:>7} -> got {n}")
asked      10 -> got 10
asked    1000 -> got 10
asked  100000 -> got 10

Ten every time. The server's cap is ten and it silently substitutes it. No amount of asking harder gets past it — only paging does.

4. Diagnose an empty result: check the CRS of your bbox

An empty result with HTTP 200 is nearly always a bounding box in the wrong units. The server has no way to know that 132000, 452000 was meant to be metres, so it interprets them in the layer's default CRS, finds nothing on earth there, and returns an empty collection.

# wrong: metres interpreted as whatever the server assumes
params = {"bbox": "132000,452000,140000,460000"}

# right: the CRS is part of the parameter
params = {"bbox": "132000,452000,140000,460000,urn:ogc:def:crs:EPSG::28992"}

For OGC API Features, bbox is WGS84 unless you pass bbox-crs, so the same numbers in metres return nothing there too.

The check that catches this before you waste an afternoon:

from shapely.geometry import box
print(box(*bbox).bounds, "->", "degrees" if max(map(abs, bbox)) <= 180 else "projected units")

5. Look for the exception report hiding behind a 200

Many OGC servers return errors as XML with a success status:

if r.text.lstrip().startswith("<"):
    message = " ".join(re.sub("<[^>]+>", " ", r.text).split())
    raise RuntimeError(f"OGC exception: {message[:200]}")
OGC exception: ows:ExceptionReport version 2.0.0 Could not locate
{http://wijkenbuurten.geonovum.nl}buurtn. Check the capabilities document …

That message names the actual problem — a misspelled layer name — which gpd.read_file() would have reported as an unhelpful GDAL parse failure.

6. Look for remark

Overpass and some other services return HTTP 200, valid JSON, a partial feature list and a top-level remark explaining that the query timed out:

if "remark" in payload:
    raise RuntimeError(f"partial result: {payload['remark']}")
Three requests asking for 10, 1000 and 100000 features, each returning the server cap of 10 with HTTP 200.
Asking for more does not produce an error. It produces the cap, again.

Code examples

Example 1 — one function that catches all four failures 

import re

import requests

HEADERS = {"User-Agent": "spatialworkflow-example/1.0 ([email protected])"}


class IncompleteResponse(RuntimeError):
    pass


def fetch_geojson(url, params=None, *, timeout=180):
    """GET a GeoJSON collection and refuse to return a silently incomplete one."""
    r = requests.get(url, params=params, headers=HEADERS, timeout=timeout)

    body = r.text.lstrip()
    if body.startswith("<"):
        message = " ".join(re.sub("<[^>]+>", " ", r.text).split())
        raise IncompleteResponse(f"HTTP {r.status_code}, XML body: {message[:200]}")
    r.raise_for_status()

    payload = r.json()
    if "remark" in payload:
        raise IncompleteResponse(f"server remark: {payload['remark']}")

    features = payload.get("features", [])
    matched = payload.get("numberMatched")
    returned = payload.get("numberReturned", len(features))

    if matched is not None and returned < matched:
        raise IncompleteResponse(f"truncated: {returned} of {matched} features")
    if not features:
        raise IncompleteResponse(f"empty result — check the bbox CRS and the filter "
                                 f"(request was {r.url})")
    return payload


try:
    fetch_geojson("https://demo.pygeoapi.io/master/collections/lakes/items",
                  {"f": "json", "limit": 100000})
except IncompleteResponse as exc:
    print(exc)
truncated: 10 of 25 features

Every branch converts a plausible-looking success into a message naming the actual problem. Note that the empty-result message includes r.url — the fully encoded request — which is the single most useful thing to have when a filter silently matches nothing.

Example 2 — paging by following the next link 

import geopandas as gpd
import pandas as pd


def fetch_all(url, params=None, *, max_pages=100):
    """Follow OGC API Features `next` links until the collection is complete."""
    collected, matched, pages = [], None, 0
    next_url, next_params = url, dict(params or {})

    while next_url and pages < max_pages:
        r = requests.get(next_url, params=next_params, headers=HEADERS, timeout=180)
        r.raise_for_status()
        payload = r.json()

        matched = payload.get("numberMatched", matched)
        collected.extend(payload.get("features", []))
        pages += 1

        links = {l["rel"]: l["href"] for l in payload.get("links", [])}
        next_url, next_params = links.get("next"), None    # the link carries its own params
        print(f"  page {pages}: {len(collected)} of {matched}")

    if matched is not None and len(collected) != matched:
        raise IncompleteResponse(f"stopped at {len(collected)} of {matched} after {pages} pages")
    return gpd.GeoDataFrame.from_features(collected, crs="EPSG:4326")


lakes = fetch_all("https://demo.pygeoapi.io/master/collections/lakes/items", {"f": "json"})
print(len(lakes), "lakes")
  page 1: 10 of 25
  page 2: 20 of 25
  page 3: 25 of 25
25 lakes

Following next beats computing offsets: the server builds the link, so it stays correct even when the server's paging rules are unusual. max_pages is a runaway guard — a misbehaving server that always returns a next link would otherwise loop forever.

Example 3 — proving the bbox units are the problem 

BASE = "https://service.pdok.nl/cbs/wijkenbuurten/2023/wfs/v1_0"

CASES = {
    "metres, no CRS declared": "132000,452000,140000,460000",
    "metres, CRS declared":    "132000,452000,140000,460000,urn:ogc:def:crs:EPSG::28992",
    "degrees, CRS declared":   "5.05,52.05,5.20,52.15,urn:ogc:def:crs:EPSG::4326",
}

for label, bbox in CASES.items():
    r = requests.get(BASE, params={
        "service": "WFS", "version": "2.0.0", "request": "GetFeature",
        "typeNames": "wijkenbuurten:buurten", "outputFormat": "application/json",
        "bbox": bbox, "count": 5,
    }, headers=HEADERS, timeout=120)
    n = len(r.json().get("features", [])) if r.text.lstrip().startswith("{") else "XML error"
    print(f"{label:26} -> {n}")
metres, no CRS declared    -> 0
metres, CRS declared       -> 5
degrees, CRS declared      -> 5

Three requests, same area, three different outcomes. The first returns zero with HTTP 200 — the exact symptom people spend hours on. Declaring the CRS is one parameter and it removes the whole class of problem.

Explanation

Why servers cap silently rather than erroring

A public spatial service is defending itself against a request for ten million features. Two responses are possible: refuse, or return a page. Both standards chose to return a page, because paging is the intended access pattern and refusing would break every naive client.

The information you need is present — numberMatched in OGC API Features, numberMatched on the WFS response element — it is just not in the part of the payload most code reads. Reading the envelope before the features is the entire discipline.

Why an empty result is almost never "there is nothing there"

Genuinely empty areas exist, but they are rare compared to the alternatives. In rough order of likelihood:

  1. The bbox is in the wrong units or CRS.
  2. The layer name is misspelled — and the server said so, in XML, with a 200.
  3. The attribute filter matches nothing because of case or an unexpected value.
  4. The axis order is swapped, putting your box in the wrong hemisphere.
  5. There is genuinely nothing there.

The way to rule out (5) is to request the same layer with no filter at all and confirm a non-zero count. If the unfiltered layer has 14,515 features and your filtered request has 0, the filter is the problem, not the data.

Why the CRS must be in the bbox parameter

A bounding box is four numbers. Nothing about 132000, 452000, 140000, 460000 says whether those are metres in a national grid, feet in a state plane, or a coordinate system nobody has heard of. The server has to assume something, and its assumption is the layer's declared default.

Both standards therefore let you say which CRS the numbers are in — a trailing URN in WFS, bbox-crs in OGC API Features. Passing it is one parameter and turns an ambiguous request into an unambiguous one.

A bounding box in national grid metres interpreted as degrees, landing far from the intended area and returning zero features.
Metres read as degrees land nowhere near the data. The server finds nothing there, correctly, and says so with HTTP 200.

Why to check both directions after any filtered download

Truncation removes features. A wrong filter removes different features. Both leave a plausible file, so check against something you know:

print(f"{len(gdf):,} features, bounds {gdf.total_bounds.round(0)}")
print(f"unique municipalities: {gdf['gemeentenaam'].nunique()}")

Bounds that are much smaller than your study area, or a nunique() of 1 when you expected 30, are both visible in one line and invisible in the feature count alone.

Edge cases or notes

  • numberMatched may be absent even when the response is complete. Some servers omit it because computing the total is expensive. Fall back to a resultType=hits request rather than assuming completeness.
  • A next link may exist on the final page on some implementations, returning an empty collection. Stop when a page yields zero features as well as when the totals agree.
  • gpd.read_file(url) hides all of this. It performs one request and gives you a GeoDataFrame with no envelope. Use it for exploration, never for a pipeline.
  • Rate limiting can also present as truncation — a 429 partway through a paged download leaves you with a partial set. Check the count after the loop, not just inside it.
  • Some services cap by response size, not feature count, so the cap moves depending on geometry complexity. A layer that returned 5,000 simple points may return 800 detailed polygons.
  • Overpass remark arrives with valid JSON and HTTP 200. See OSMnx download fails, hangs or times out for how easily that one hides.
  • Log the full request URL on failure. response.url includes the encoded parameters, and it is usually enough to spot the problem without reproducing it.

FAQ

Why did I get exactly 1,000 features?

That is a server-side cap. Round numbers are almost never coincidences. Find the true total with numberMatched or a resultType=hits request, then page.

I asked for a bigger limit and still got the same number. Why?

Servers clamp limit to their maximum and return it without an error. Paging is the only way past it.

Why does my bbox return zero features?

Nearly always units. Declare the CRS of your bounding box — a trailing URN in WFS, bbox-crs in OGC API Features — so the numbers are interpreted as you intended.

The response is HTTP 200 but GeoPandas cannot read it. What is it?

An XML exception report. Check whether the body starts with < and strip the tags to read the server's actual message.

What is remark in a GeoJSON response?

A server-side timeout notice. The features present are a partial result. Treat any response containing it as a failure.

How do I know my download is complete?

Compare what you received against the count the server reports, and raise when they disagree. That comparison is the only reliable completeness check available.

Is gpd.read_file() on a service URL ever safe?

For exploring, yes. For anything whose count matters, no — it performs a single unpaged request and discards the envelope that would have told you the result was short.