Null, Empty, Missing and Invalid: Four Kinds of Broken Geometry

Problem statement

The layer is the Isle of Wight's 86,080 OpenStreetMap multipolygons — landuse, buildings, woods, the lot. dropna() removes nothing. The analysis still loses rows.

gdf = features.dropna(subset=["geometry"])
core = gdf.assign(geometry=gdf.buffer(-5))     # each feature's interior, 5 m in from its edge

joined = gpd.sjoin(core, wards, predicate="within")
print(f"{len(features):,} rows -> dropna {len(gdf):,} -> sjoin {len(joined):,}")
print(f"area: {gdf.area.sum() / 1e6:,.1f} km² before the buffer, {core.area.sum() / 1e6:,.1f} km² after")

86,080 rows -> dropna 86,080 -> sjoin 21,703
area: 1,408.5 km² before the buffer, 1,356.4 km² after

Tens of thousands of rows silently matched nothing, and dropna() reported that it had cleaned the data. Some of those rows straddle a ward boundary, which is what within is for. Most of them are something else.

The problem is that "broken geometry" is not one condition. It is four different conditions, each detected by a different check, each with a different consequence, and only one of them is caught by dropna().

Quick answer

Four states, four checks, four fixes:

def geometry_health(gdf):
    return {
        "rows":    len(gdf),
        "null":    int(gdf.geometry.isna().sum()),
        "empty":   int(gdf.geometry.is_empty.sum()),
        "invalid": int((~gdf.is_valid & gdf.geometry.notna()).sum()),
        "no_crs":  gdf.crs is None,
    }

print("features:", geometry_health(features))
print("core:    ", geometry_health(core))
print("food:    ", geometry_health(food))

features: {'rows': 86080, 'null': 0, 'empty': 0, 'invalid': 19, 'no_crs': False}
core:     {'rows': 86080, 'null': 0, 'empty': 63548, 'invalid': 0, 'no_crs': False}
food:     {'rows': 1701, 'null': 375, 'empty': 0, 'invalid': 0, 'no_crs': False}

The empties are most of the answer to the missing rows: they are not null, so dropna() left them, and they have no coordinates, so they are within no ward. The source layer carries 19 invalid polygons of its own, and the food register — the Food Standards Agency's list of the island's food businesses — has rows with no location at all. The invalid count leaves nulls out on purpose: is_valid is False for a null, so a plain ~is_valid counts every null twice over.

# the check that actually cleans
cleaned = core[core.geometry.notna() & ~core.geometry.is_empty].copy()
cleaned["geometry"] = cleaned.geometry.make_valid()
print(geometry_health(cleaned))

{'rows': 22532, 'null': 0, 'empty': 0, 'invalid': 0, 'no_crs': False}

The four states side by side

Null, empty, missing, invalid, in that order

1. Null geometry — the row has no shape at all

print(food.geometry.isna().sum(), "of", len(food), "food businesses have no geometry")

375 of 1701 food businesses have no geometry

In GeoPandas a null geometry is None, stored in the geometry column like any missing value. It happens when a source row had no geometry field, when a join brought in unmatched rows, or when a transformation failed for a single feature — here, when the register publishes a business without coordinates.

print(food[food.geometry.isna()][["BusinessName", "BusinessType", "PostCode"]].head(4).to_string(index=False))

                             BusinessName            BusinessType PostCode
Age Concern Shanklin - Heather Whitchurch Other catering premises     PO37
         Antoine's Little Pudding Parlour Restaurant/Cafe/Canteen     PO37
                         Aramark @ CECAMM Restaurant/Cafe/Canteen PO32 6FY
                                 Archie's  Takeaway/sandwich shop     PO35

If the attributes are intact, the row is a real record with a lost shape — worth investigating rather than deleting. These have names and postcodes, so a geocoder could give them back a location.

located = food[food.geometry.notna()]

2. Empty geometry — a shape with nothing in it

from shapely.geometry import Polygon
empty = Polygon()

print(empty is None, empty.is_empty, empty.is_valid, empty.area, empty.geom_type)

False True True 0.0 Polygon

An empty geometry passes every check people usually run. It is not null, it is valid, it has a geometry type, and it writes to a file without complaint. It also intersects nothing, contains nothing, and contributes zero to every sum.

Empties are produced by operations far more often than they are read from files. On the island's features — the source layer has none:

newport = wards.loc[wards["WD24NM"].str.startswith("Newport"), "geometry"].union_all()
made = {
    "intersection with Newport": gdf.intersection(newport),
    "buffer(-5)": gdf.buffer(-5),
    "buffer(-50)": gdf.buffer(-50),
}
for label, result in made.items():
    print(f"{label:26} {result.is_empty.sum():>7,} empty of {len(result):,}")

intersection with Newport   83,143 empty of 86,080
buffer(-5)                  63,548 empty of 86,080
buffer(-50)                 83,924 empty of 86,080

The negative buffer is the common trap: every feature thinner than twice the distance disappears, and almost all buildings are thinner than 100 m.

before = len(core)
core = core[~core.geometry.is_empty]
print(f"dropped {before - len(core):,} empty geometries")

dropped 63,548 empty geometries

3. Missing attributes — the geometry is fine, the data is not

print(gdf[["name", "landuse", "natural", "building"]].isna().sum().to_dict())

{'name': 83278, 'landuse': 81096, 'natural': 80755, 'building': 15075}

Attribute nulls do not break spatial operations, so they survive every geometry check and then quietly change results:

import math

areas = gdf.assign(area_m2=gdf.area)
by_landuse = areas.groupby("landuse")["area_m2"].sum()      # null landuse drops out of the result
print(f"grouped total {by_landuse.sum() / 1e6:,.1f} km² vs layer total {areas['area_m2'].sum() / 1e6:,.1f} km²")

areas["landuse"] = areas["landuse"].fillna("UNKNOWN")        # make the absence visible
totals = areas.groupby("landuse")["area_m2"].sum()
assert math.isclose(totals.sum(), areas["area_m2"].sum())
print(f"with UNKNOWN: {totals.sum() / 1e6:,.1f} km², of which UNKNOWN {totals['UNKNOWN'] / 1e6:,.1f}")

grouped total 300.3 km² vs layer total 1,408.5 km²
with UNKNOWN: 1,408.5 km², of which UNKNOWN 1,108.2

groupby drops null keys by default. The total no longer reconciles with the layer total, and nothing said so. See how to handle missing and null values in spatial datasets for the full treatment.

4. Invalid geometry — the shape breaks the rules

from shapely.validation import explain_validity

invalid = features[~features.is_valid]
print(len(invalid), invalid.geometry.map(lambda g: explain_validity(g).split("[")[0]).value_counts().to_dict())

19 {'Self-intersection': 19}

repaired = invalid.geometry.make_valid()
print(repaired.is_valid.all(), repaired.geom_type.value_counts().to_dict())

True {'GeometryCollection': 19}

make_valid can change the geometry type — every one of the island's 19 came back as a GeometryCollection, and degenerate shapes can come back as lines or points. Keep the part you want:

polygons_only = repaired.map(lambda g: shapely.union_all([p for p in getattr(g, "geoms", [g])
                                                            if p.geom_type in ("Polygon", "MultiPolygon")]))
print(polygons_only.geom_type.value_counts().to_dict())

{'Polygon': 15, 'MultiPolygon': 4}

See how to fix invalid geometries and buffer(0) does not fix my invalid geometry.

5. Do them in the right order

Order matters, because each step changes what the next one sees:

def clean_geometry(gdf, report=None):
    r = report if report is not None else {}
    r["in"] = len(gdf)

    gdf = gdf[gdf.geometry.notna()]                 # 1. null — nothing to repair
    r["dropped_null"] = r["in"] - len(gdf)

    n = len(gdf)
    gdf = gdf[~gdf.geometry.is_empty]               # 2. empty — before validity
    r["dropped_empty"] = n - len(gdf)

    bad = (~gdf.is_valid).sum()                     # 3. invalid — repair, do not drop
    if bad:
        gdf = gdf.assign(geometry=gdf.geometry.make_valid())
        r["repaired"] = int(bad)

    n = len(gdf)                                    # 4. check again after the repair
    gdf = gdf[~gdf.geometry.is_empty]
    r["empty_after_repair"] = n - len(gdf)

    r["out"] = len(gdf)
    return gdf, r

_, report = clean_geometry(gdf.assign(geometry=gdf.buffer(-5)))
print(report)

{'in': 86080, 'dropped_null': 0, 'dropped_empty': 63548, 'empty_after_repair': 0, 'out': 22532}

Step 4 is cheap insurance. Here it had nothing to do: the buffer had already rebuilt the 19 invalid shapes as valid ones, so there was nothing to repair. But a repair is a geometry operation like any other, and the empty check after it costs one line.

A geometry health report for Isle of Wight layers

The data used on this page

Three real layers for the Isle of Wight: every multipolygon in Geofabrik's OpenStreetMap extract from its dated 1 January 2026 snapshot, the island's wards from the ONS (December 2024, full resolution), and the Food Standards Agency's food hygiene register for the island (local authority 284) from its public API, with a point for each business that has coordinates. All in British National Grid. The food register is a copy of the API's answer saved on 10 October 2026.

from pathlib import Path

import geopandas as gpd
import pandas as pd
import requests
import shapely
from shapely.geometry import Point

PBF = Path("isle-of-wight-260101.osm.pbf")
if not PBF.exists():
    reply = requests.get("https://download.geofabrik.de/europe/united-kingdom/england/"
                         "isle-of-wight-260101.osm.pbf", timeout=300)
    reply.raise_for_status()
    PBF.write_bytes(reply.content)
features = (gpd.read_file(PBF, layer="multipolygons")
            [["osm_id", "name", "landuse", "natural", "building", "geometry"]].to_crs(27700))

if not Path("iow_wards.gpkg").exists():
    reply = requests.get("https://services1.arcgis.com/ESMARspQHYMw9BZ9/arcgis/rest/services/"
                         "Wards_December_2024_Boundaries_UK_BFC/FeatureServer/0/query",
                         params={"where": "LAD24NM='Isle of Wight'", "outFields": "WD24CD,WD24NM",
                                 "outSR": 27700, "f": "geojson"}, timeout=120)
    reply.raise_for_status()
    gpd.read_file(reply.text).to_file("iow_wards.gpkg")
wards = gpd.read_file("iow_wards.gpkg")

if not Path("iow_food.gpkg").exists():
    # live: requests.get("https://api.ratings.food.gov.uk/Establishments", headers={"x-api-version": "2"},
    #                    params={"localAuthorityId": 284, "pageSize": 5000}, timeout=300)   # Isle of Wight
    # read instead as the API answered on 10 October 2026, so the register stays put
    reply = requests.get("https://data.spatialworkflow.io/fhrs-api-284-2026-10-10.json", timeout=300)
    reply.raise_for_status()
    rows = pd.json_normalize(reply.json()["establishments"])
    lon = pd.to_numeric(rows["geocode.longitude"], errors="coerce")
    lat = pd.to_numeric(rows["geocode.latitude"], errors="coerce")
    geometry = [Point(x, y) if pd.notna(x) and pd.notna(y) else None for x, y in zip(lon, lat)]
    (gpd.GeoDataFrame(rows[["FHRSID", "BusinessName", "BusinessType", "PostCode"]], geometry=geometry, crs=4326)
        .to_crs(27700).to_file("iow_food.gpkg"))
food = gpd.read_file("iow_food.gpkg")

print(f"{len(features):,} OSM multipolygons, {len(wards)} wards, {len(food):,} food businesses")

86,080 OSM multipolygons, 39 wards, 1,701 food businesses

Example 1: a health report you can put in a log

from shapely.validation import explain_validity

def geometry_report(gdf, name="layer"):
    present = gdf[gdf.geometry.notna()]
    invalid = present[~present.is_valid]
    reasons = (
        invalid.geometry.apply(lambda g: explain_validity(g).split("[")[0])
        .value_counts().to_dict() if len(invalid) else {}
    )
    return {
        "layer": name,
        "rows": len(gdf),
        "crs": str(gdf.crs),
        "null": int(gdf.geometry.isna().sum()),
        "empty": int(gdf.geometry.is_empty.sum()),
        "invalid": len(invalid),
        "invalid_reasons": reasons,
        "types": present.geom_type.value_counts().to_dict(),
    }

print(geometry_report(features, "iow multipolygons"))
print(geometry_report(food, "iow food businesses"))

{'layer': 'iow multipolygons', 'rows': 86080, 'crs': 'EPSG:27700', 'null': 0, 'empty': 0, 'invalid': 19, 'invalid_reasons': {'Self-intersection': 19}, 'types': {'MultiPolygon': 86080}}
{'layer': 'iow food businesses', 'rows': 1701, 'crs': 'EPSG:27700', 'null': 375, 'empty': 0, 'invalid': 0, 'invalid_reasons': {}, 'types': {'Point': 1326}}

One dict, JSON-serialisable, worth writing to the run log of every job. When a total looks wrong three months later, this is the record that says whether the input was already broken.

Example 2: catching empties the moment an operation creates them

def clip_checked(gdf, boundary):
    out = gdf.clip(boundary)
    empties = out.geometry.is_empty.sum()
    if empties:
        print(f"clip produced {empties} empty geometries of {len(out)}")
        out = out[~out.geometry.is_empty]
    return out

clipped = clip_checked(features, newport)
print(f"{len(features):,} features -> {len(clipped):,} inside Newport")

86,080 features -> 2,937 inside Newport

clip itself does not hand back empties: it drops the features that fall outside the boundary, so the row count is what changes — 86,080 in, a few thousand out, and no empties to report. intersection, difference, negative buffer, and simplify with an aggressive tolerance do keep the row and empty the geometry, which is where the check earns its place.

Example 3: asserting the invariant instead of hoping

def assert_usable(gdf, name="layer"):
    problems = []
    if gdf.geometry.isna().any():
        problems.append(f"{gdf.geometry.isna().sum()} null geometries")
    if gdf.geometry.is_empty.any():
        problems.append(f"{gdf.geometry.is_empty.sum()} empty geometries")
    present = gdf.geometry.notna()
    if not gdf[present].is_valid.all():
        problems.append(f"{(~gdf[present].is_valid).sum()} invalid geometries")
    if gdf.crs is None:
        problems.append("no CRS")
    if problems:
        raise ValueError(f"{name} is not usable: " + "; ".join(problems))

for name, layer in [("features", features), ("food", food), ("cleaned", cleaned)]:
    try:
        assert_usable(layer, name)
        print(f"{name}: usable")
    except ValueError as exc:
        print(exc)

features is not usable: 19 invalid geometries
food is not usable: 375 null geometries
cleaned: usable

Call it after loading and after every step that can produce empties. An exception at the point of damage beats a wrong number at the end.

Example 4: what each file format does with null and empty

from shapely.geometry import Point

probe = gpd.GeoDataFrame({"state": ["point", "null", "empty"]},
                         geometry=[Point(450000, 90000), None, Point()], crs=27700)
for driver, ext in [("ESRI Shapefile", "shp"), ("GeoJSON", "geojson"), ("GPKG", "gpkg")]:
    probe.to_file(f"probe.{ext}", driver=driver)
    back = gpd.read_file(f"probe.{ext}")
    print(f"{driver:15} null -> {'null' if back.geometry[1] is None else 'empty'}, "
          f"empty -> {'null' if back.geometry[2] is None else 'empty'}")

ESRI Shapefile  null -> null, empty -> null
GeoJSON         null -> null, empty -> null
GPKG            null -> null, empty -> empty

A GeoPackage keeps the two states apart. A shapefile and GeoJSON both write an empty geometry as null, so after a round trip through either, an is_empty check finds nothing and an isna check finds the empties.

Four states at different layers of the stack

The reason these four states are so easy to confuse is that they live at different layers of the stack.

Null is a pandas concept. The geometry column is a column like any other, and pandas allows missing values in it. dropna(), fillna() and isna() all work — and only see this layer.

Empty is a Shapely and OGC concept. Polygon() with no coordinates is a well-defined object in the Simple Features model: it has a type, it is valid, and its point set is the empty set. Everything follows logically from that — it intersects nothing because there is nothing to intersect, its area is zero because it encloses nothing. Pandas has no opinion about it, because to pandas it is just an object in a cell.

Invalid is a third layer, GEOS's, and missing attributes a fourth that has nothing to do with geometry; no check at one layer sees the others.

is_empty on nulls, valid empties and format behaviour

• is_empty on a null geometry returns False, and so does is_valid. A null is neither empty nor valid, so filter nulls first, then empties, and do not read ~is_valid as a count of broken shapes while nulls are present.
• gdf.explode() keeps an empty geometry as its own row, while splitting multi-parts into several — so row counts change for two reasons, and only one of them is data.
• An all-empty layer still has a CRS and a schema, so it writes a valid file with zero usable features. Assert on len() after filtering.
• unary_union of a set containing empties works fine — they contribute nothing, which is the mathematically correct behaviour and occasionally a surprise.

Internal links

• How to fix invalid geometries in Python (GeoPandas) — repairing state four
• buffer(0) does not fix my invalid geometry — when make_valid is the only answer
• How to remove null and empty geometries in GeoPandas — the practical filter
• How to handle missing and null values in spatial datasets — state three in depth
• What makes a geometry valid? The OGC rules explained — where the validity rules come from
• Spatial data quality: the six dimensions that matter — the wider frame
• What to test in a GIS pipeline — asserting these invariants automatically
• The Python GIS data cleaning checklist — where this fits in a full clean

FAQ

Why does dropna() not remove empty geometries?

Because an empty geometry is not missing. It is a real Shapely object with a type and a validity state; its point set just happens to be empty. Use ~gdf.geometry.is_empty as a separate filter.

Is an empty geometry valid?

Yes. Polygon().is_valid is True. Validity asks whether the shape breaks the OGC rules; an empty shape breaks none of them.

What creates empty geometries?

Almost always an operation rather than a file: clip, intersection and difference on non-overlapping inputs, negative buffer on narrow shapes, and aggressive simplify.

Should I drop invalid geometries or repair them?

Repair with make_valid, then check the result. Dropping loses real features, and the underlying problem is usually a digitising artefact rather than a wrong record.

How do I know why a geometry is invalid?

from shapely.validation import explain_validity returns a reason and the coordinate where it occurs — useful for finding a systematic problem in one source.

Does an empty geometry break a spatial join?

Not with an error. It simply matches nothing, so the row disappears from an inner join and gets nulls in a left join. That silence is exactly the problem.

What is the minimum check before analysis?

notna(), ~is_empty, is_valid.all() and crs is not None. Four lines, and they cover every state on this page.