How to Validate a GeoDataFrame Against a Schema Before Analysis
On this page
Problem statement
Spatial code rarely crashes on bad input; it produces wrong output. A renamed column empties a groupby, a layer in degrees returns areas in square degrees, rows without geometry fall out of a join, and the pipeline runs green the whole way. The defence is a written contract β which columns, which types, which CRS, which geometry, which values β checked in one pass before analysis, with every failure reported at once.
This page builds that check in plain GeoPandas and runs it on a dataset with real problems: the Food Standards Agency's register of Rutland's 462 food businesses, 124 of them published without a location and with ratings that are sometimes a number and sometimes the word Exempt. Two of the problems it finds are invisible to the checks people usually write.
Quick answer
Define your expectations as a plain dict, then run one function that checks required columns, dtypes, CRS, and geometry validity, collecting every failure into a list. If the list is non-empty, raise before any analysis runs.
import geopandas as gpd
import pandas as pd
def validate(gdf, schema):
issues = []
# Required columns
missing = [c for c in schema.get("required_columns", []) if c not in gdf.columns]
for c in missing:
issues.append(f"missing required column: {c}")
# Dtypes
for col, expected in schema.get("dtypes", {}).items():
if col in gdf.columns and not pd.api.types.is_dtype_equal(gdf[col].dtype, expected):
issues.append(f"column '{col}' has dtype {gdf[col].dtype}, expected {expected}")
# CRS
expected_crs = schema.get("crs")
if expected_crs is not None:
if gdf.crs is None:
issues.append("CRS is missing (None)")
elif gdf.crs.to_epsg() != expected_crs:
issues.append(f"CRS is EPSG:{gdf.crs.to_epsg()}, expected EPSG:{expected_crs}")
# Geometry validity and null geometries
if gdf.geometry.isna().any():
issues.append(f"{int(gdf.geometry.isna().sum())} null geometries")
invalid = ~gdf.geometry.is_valid & gdf.geometry.notna()
if invalid.any():
issues.append(f"{int(invalid.sum())} invalid geometries")
return issues
schema = {"required_columns": ["FHRSID", "rating"], "dtypes": {"FHRSID": "int64"}, "crs": 27700}
problems = validate(register, schema)
try:
if problems:
raise ValueError("GeoDataFrame failed validation:\n" + "\n".join(problems))
except ValueError as exc:
print(exc)
print("null geometries:", register.geometry.isna().sum())
GeoDataFrame failed validation:
missing required column: rating
CRS is EPSG:4326, expected EPSG:27700
124 invalid geometries
null geometries: 0
The register as published fails three ways β and the null check reports nothing wrong. Its 124 unlocated businesses are not null: points_from_xy turned their missing coordinates into POINT (NaN NaN), which only the validity check catches.
A schema, and one check per concern
The plan: describe your expectations in a schema dict, write one check per concern, and have each check append human-readable strings to a shared issues list. At the end you decide whether to raise.
Define the schema
A plain dict, one key per kind of expectation, kept in version control beside the code:
schema = {
"required_columns": ["FHRSID", "BusinessName", "BusinessType", "rating", "RatingDate", "geometry"],
"dtypes": {
"FHRSID": "int64",
"BusinessName": "str",
"rating": "Int64",
},
"crs": 27700, # British National Grid
"geometry_type": "Point",
"ranges": {
"rating": (0, 5), # the FHRS scale, inclusive
"hygiene": (0, 25), # the hygiene score, 0 best
},
"unique": ["FHRSID"],
"non_null": ["FHRSID", "BusinessName"],
}
Every check below reads from one of these keys. If a key is absent, the corresponding check is simply skipped, so you can start small and add rules over time.
Check required columns and dtypes
First confirm the columns exist, then check the dtype only for columns that are present (checking the dtype of a missing column would throw).
def check_columns(gdf, schema, issues):
for col in schema.get("required_columns", []):
if col not in gdf.columns:
issues.append(f"missing required column: {col}")
for col, expected in schema.get("dtypes", {}).items():
if col not in gdf.columns:
continue # already reported as missing
actual = gdf[col].dtype
if not pd.api.types.is_dtype_equal(actual, expected):
issues.append(
f"column '{col}' has dtype {actual}, expected {expected}"
)
return issues
Using pd.api.types.is_dtype_equal is more robust than gdf[col].dtype == expected because it handles string aliases like "int64" and extension dtypes consistently. It is also exact, which bites on a pandas upgrade:
print(pd.__version__, register["BusinessName"].dtype)
print("matches 'object':", pd.api.types.is_dtype_equal(register["BusinessName"].dtype, "object"))
print("matches 'str': ", pd.api.types.is_dtype_equal(register["BusinessName"].dtype, "str"))
3.0.6 str
matches 'object': False
matches 'str': True
pandas 3 reads text as its str dtype, so a schema written for pandas 2 that expects "object" for every text column fails on every one of them after the upgrade β with no change to the data.
Check the CRS
Compare by EPSG code rather than by string, so equivalent definitions still match:
def check_crs(gdf, schema, issues):
expected = schema.get("crs")
if expected is None:
return issues # no CRS requirement declared
if gdf.crs is None:
issues.append("CRS is missing (None)")
return issues
actual = gdf.crs.to_epsg()
if actual != expected:
issues.append(f"CRS is EPSG:{actual}, expected EPSG:{expected}")
return issues
gdf.crs.to_epsg() returns the integer EPSG code (or None if the CRS cannot be reduced to one). Comparing integers avoids the trap where "EPSG:4326" and a full WKT string describe the same system but compare unequal as text.
Check geometry type and validity
Type, nulls and validity are three separate checks, so the report says which one failed β and on this register only one of them catches the unlocated businesses:
def check_geometry(gdf, schema, issues):
geom = gdf.geometry
null_count = int(geom.isna().sum())
if null_count:
issues.append(f"{null_count} null geometries")
empty_count = int((geom.notna() & geom.is_empty).sum())
if empty_count:
issues.append(f"{empty_count} empty geometries")
present = geom[geom.notna() & ~geom.is_empty]
expected_type = schema.get("geometry_type")
if expected_type is not None:
bad_type = present.geom_type != expected_type
if bad_type.any():
found = sorted(present.geom_type.unique().tolist())
issues.append(
f"{int(bad_type.sum())} geometries are not {expected_type} "
f"(found types: {found})"
)
invalid = ~present.is_valid
if invalid.any():
issues.append(f"{int(invalid.sum())} invalid geometries")
return issues
Check value ranges and uniqueness
Numeric columns often have implicit bounds β a population is never negative, a percentage sits between 0 and 100. Identifier columns must be unique, and some columns must never be null.
def check_values(gdf, schema, issues):
for col, (low, high) in schema.get("ranges", {}).items():
if col not in gdf.columns:
continue
s = gdf[col]
out = s[(s < low) | (s > high)]
if len(out):
issues.append(
f"column '{col}' has {len(out)} values outside [{low}, {high}]"
)
for col in schema.get("unique", []):
if col not in gdf.columns:
continue
dups = int(gdf[col].duplicated().sum())
if dups:
issues.append(f"column '{col}' has {dups} duplicate values")
for col in schema.get("non_null", []):
if col not in gdf.columns:
continue
nulls = int(gdf[col].isna().sum())
if nulls:
issues.append(f"column '{col}' has {nulls} null values")
return issues
Collect issues into a report and fail fast
Now wire the checks together. Each appends to the same list, so you get every problem in one run instead of fixing them one at a time.
def validate(gdf, schema):
issues = []
check_columns(gdf, schema, issues)
check_crs(gdf, schema, issues)
check_geometry(gdf, schema, issues)
check_values(gdf, schema, issues)
return issues
def validate_or_raise(gdf, schema):
issues = validate(gdf, schema)
if issues:
report = "\n".join(f" - {i}" for i in issues)
raise ValueError(f"GeoDataFrame failed validation:\n{report}")
return gdf
for name, frame in (("as published", register), ("cleaned", clean_register)):
try:
validate_or_raise(frame, schema)
print(f"{name}: passed, {len(frame)} rows")
except ValueError as exc:
print(f"{name}: {exc}")
as published: GeoDataFrame failed validation:
- missing required column: rating
- CRS is EPSG:4326, expected EPSG:27700
- 124 invalid geometries
cleaned: passed, 338 rows
The published register fails three ways in one run, which is the point of collecting them: the missing numeric rating, the CRS and the unlocated businesses are all visible at once, instead of one per re-run. Checks on columns that are absent are skipped rather than reported twice, which is why the rating's dtype and range do not appear. clean_register β the setup's clean(): drop the unlocated rows, turn the rating into a number, parse the date, reproject β passes. The cleaning step is where the fixes live; validation only proves they happened.
Validating Rutland's food register and wards
The data used on this page
The Food Standards Agency's food hygiene rating data for Rutland (Open Government Licence) and Rutland's 15 wards from the ONS December 2024 boundaries (OGL).
import json
import warnings
from pathlib import Path
import geopandas as gpd
import pandas as pd
import requests
Path("data").mkdir(exist_ok=True)
if not Path("data/food.json").exists():
Path("data/food.json").write_bytes(requests.get(
# the FSA register file for Rutland as saved on 9 October 2026 (updated daily); live:
# ratings.food.gov.uk/api/open-data-files/FHRS879en-GB.json
"https://data.spatialworkflow.io/fhrs-879-2026-10-09.json", timeout=120).content)
if not Path("data/wards.geojson").exists():
Path("data/wards.geojson").write_bytes(requests.get(
"https://services1.arcgis.com/ESMARspQHYMw9BZ9/arcgis/rest/services/"
"Wards_December_2024_Boundaries_UK_BFC/FeatureServer/0/query",
params={"where": "LAD24CD='E06000017'", "outFields": "WD24NM", "outSR": 27700, "f": "geojson"},
timeout=120).content)
records = pd.json_normalize(json.loads(Path("data/food.json").read_text())["FHRSEstablishment"]["EstablishmentCollection"])
x = pd.to_numeric(records["Geocode.Longitude"], errors="coerce")
y = pd.to_numeric(records["Geocode.Latitude"], errors="coerce")
register = gpd.GeoDataFrame(records[["FHRSID", "BusinessName", "BusinessType", "RatingValue", "RatingDate", "Scores.Hygiene"]],
geometry=gpd.points_from_xy(x, y), crs=4326)
def clean(gdf):
located = gdf[gdf.geometry.is_valid & ~gdf.geometry.is_empty]
return (located.assign(rating=pd.to_numeric(located["RatingValue"], errors="coerce").astype("Int64"),
RatingDate=pd.to_datetime(located["RatingDate"], errors="coerce"),
hygiene=located["Scores.Hygiene"])
.drop(columns=["RatingValue", "Scores.Hygiene"])
.to_crs(27700))
clean_register = clean(register)
print(f"{len(register)} businesses published, {len(clean_register)} located;",
register["RatingValue"].value_counts().to_dict())
462 businesses published, 338 located; {'5': 354, '4': 39, 'Exempt': 27, 'AwaitingInspection': 25, '3': 10, '2': 4, '1': 3}
Example 1: Minimal validation at load time
ward_schema = {"required_columns": ["WD24NM", "geometry"], "crs": 27700, "unique": ["WD24NM"]}
wards = gpd.read_file("data/wards.geojson")
problems = validate(wards, ward_schema)
print(problems or f"wards: OK ({len(wards)} rows)")
wards: OK (15 rows)
Example 2: Warn instead of raise
During exploration you may want to see issues without stopping. Log them and continue.
import warnings
def validate_or_warn(gdf, schema):
issues = validate(gdf, schema)
for i in issues:
warnings.warn(f"validation: {i}")
return issues
with warnings.catch_warnings(record=True) as caught:
warnings.simplefilter("always")
found = validate_or_warn(register, schema)
print(f"{len(caught)} warnings, first: {caught[0].message}")
3 warnings, first: validation: missing required column: rating
Example 3: Validating a non-geometry concern only
The schema keys are independent, so you can run a partial schema β useful in unit tests for a single rule.
dates_only = {"non_null": ["RatingDate"]}
print("published:", validate(register, dates_only))
print("cleaned: ", validate(clean_register, dates_only))
print(register.loc[register["RatingDate"].isna(), "RatingValue"].value_counts().to_dict())
published: ['124 invalid geometries', "column 'RatingDate' has 52 null values"]
cleaned: ["column 'RatingDate' has 38 null values"]
{'Exempt': 27, 'AwaitingInspection': 25}
A one-rule schema is also how you find out whether a rule is right. The missing dates all belong to businesses that are AwaitingInspection or Exempt β businesses that have no rating yet cannot have a rating date. RatingDate is optional by design, which is why it is not in the main schema's non_null list.
Example 4: Checking for an expected coordinate bounds box
A cheap sanity check: do the geometries fall inside the rough bounding box you expect for your study area? This catches data accidentally in the wrong CRS even when .crs is set correctly.
def check_bounds(gdf, expected_bbox, issues):
minx, miny, maxx, maxy = gdf.total_bounds
e_minx, e_miny, e_maxx, e_maxy = expected_bbox
if minx < e_minx or miny < e_miny or maxx > e_maxx or maxy > e_maxy:
issues.append(
f"data bounds {tuple(round(float(b)) for b in gdf.total_bounds)} "
f"fall outside expected {expected_bbox}"
)
return issues
rutland = tuple(round(float(v)) for v in wards.total_bounds) # the county, in BNG metres
print("cleaned:", check_bounds(clean_register, rutland, []))
# the classic slip: latitude passed as x, longitude as y, still labelled EPSG:4326
located = register[register.geometry.is_valid]
swapped = gpd.GeoDataFrame(located.drop(columns="geometry"),
geometry=gpd.points_from_xy(located.geometry.y, located.geometry.x), crs=4326)
print("CRS check:", validate(swapped, {"crs": 4326}))
print("swapped:", check_bounds(swapped.to_crs(27700), rutland, []))
cleaned: []
CRS check: []
swapped: ['data bounds (7680773, -5680034, 7722808, -5614508) fall outside expected (479688, 292648, 506296, 318928)']
The swapped layer passes the CRS check β its label really is EPSG:4326 β and lands in the Indian Ocean, at 52.7Β° E, 0.7Β° S. Only a check against where the data should be catches it.
Example 5: Reusing the validator across multiple datasets
Keep one schema per dataset type and loop over inputs in a batch job.
schemas = {
"register": schema,
"wards": {"required_columns": ["WD24NM"], "crs": 27700,
"geometry_type": "Polygon", "unique": ["WD24NM"]},
}
for name, frame in (("register", clean_register), ("wards", wards)):
problems = validate(frame, schemas[name])
print(f"{name}: {problems or 'OK'} ({len(frame)} rows)")
register: OK (338 rows)
wards: OK (15 rows)
Why the usual checks miss the worst problems
The checks people write first β is the column there, is the geometry null β test the form of the data, and the register's worst problems have the right form. Its 124 unlocated businesses are not null: building points from empty coordinate columns gives POINT (NaN NaN), a geometry object that exists, so isna() passes it and only is_valid objects. A layer with latitude and longitude swapped carries a perfectly correct EPSG:4326 label, so the CRS check passes it too, and only a bounding box for where the data ought to be (Example 4) puts it in the Indian Ocean. And a schema can fail with no change to the data at all: pandas 3 reads text as str, so every rule written for pandas 2's object breaks on upgrade.
That is the case for collecting every failure before raising. The first run against the published register reported three independent problems β the missing numeric rating, the CRS, the unlocated businesses β and a validator that stopped at the first would have taken three runs to say so. Validation belongs between cleaning and analysis: it proves the cleaning happened, and the cleaning is where the fixes live.
Mixed types, float bounds and bigger tools
Geometry type can legitimately be mixed
Real datasets often mix Polygon and MultiPolygon, or LineString and MultiLineString, and that is perfectly valid. If your analysis handles both, allow a set of types rather than a single string:
allowed = {"Polygon", "MultiPolygon"}
bad = ~present.geom_type.isin(allowed)
Decide deliberately whether your downstream code cares about the distinction before you reject mixed types.
Numeric tolerance for ranges
Floating-point columns rarely land on exact bounds. A "percentage" stored as a float might read 100.0000001 after a computation. If strict bounds cause spurious failures, widen the range by a small epsilon or round before comparing:
eps = 1e-9
out = s[(s < low - eps) | (s > high + eps)]
For integer columns this is unnecessary; reserve tolerance for floats.
Larger projects
pandera declares a typed schema, can validate a GeoDataFrame including its geometry and CRS, and produces structured error reports; Great Expectations adds monitoring across whole pipelines. Both are worth their dependencies once the schema is shared and long-lived; for a script, the dict above is enough.
Internal links
- What Is a GeoDataFrame? Structure and Key Concepts
- The Python GIS Data Cleaning Checklist
- How to Fix CRS Mismatch in GeoPandas
- How to Find and Remove Duplicate Geometries in GeoPandas
- How to Clean and Normalise Attribute Columns in a GeoDataFrame
FAQ
Should validation raise an exception or return a list?
Do both, in layers. Have validate() return a list of issue strings so it is easy to test, log, or inspect. Then wrap it in a thin validate_or_raise() that raises when the list is non-empty. This separation lets you choose strict failure in production and a softer warn-and-continue mode during exploration.
How do I validate that geometries are valid?
Use the gdf.geometry.is_valid boolean Series, which flags self-intersections and malformed geometries. Count the False entries and report them. To inspect why, gdf.geometry.make_valid() can repair them in your cleaning stage, but keep repair out of the validator itself.
Can I check column dtypes loosely instead of exactly?
Yes. For numeric columns you often care that a value is numeric, not that it is exactly int64. Use pd.api.types.is_numeric_dtype(gdf[col]) instead of an exact match. Define helper predicates in your schema when "any integer" or "any float" is good enough, and reserve exact dtype checks for cases where the storage type genuinely matters.
How is this different from just using assert statements?
A chain of assert statements stops at the first failure, so you discover problems one re-run at a time. The collect-into-a-list pattern reports every issue in a single pass and gives you readable messages rather than a bare AssertionError. It is the same idea as a linter: surface all the problems at once.
When should I switch from this to pandera?
Switch when validation stops being a few lines at the top of a script and becomes infrastructure β shared across modules, run in CI, or maintained by a team. pandera gives you declarative typed schemas, GeoDataFrame support for geometry and CRS rules, and structured error objects. For a single analysis script, the plain-dict approach is lighter and just as effective.
Does validation slow down my pipeline?
The checks here are vectorized pandas and GeoPandas operations, so they are fast relative to the analysis that follows. The most expensive check is usually is_valid on large geometry columns, which is still cheap compared to a spatial join or overlay. The time you spend validating is far less than the time you would spend debugging a silently wrong result.