🧭 Concept Data cleaning Geometry & topology Updated

Null, Empty, Missing and Invalid: Four Kinds of Broken Geometry

Problem statement

The layer has 12,400 rows. dropna() removes 6. The analysis still fails.

gdf = gpd.read_file("parcels.gpkg")
gdf = gdf.dropna(subset=["geometry"])          # 12,394 rows

joined = gpd.sjoin(gdf, wards, predicate="within")
print(len(joined))                              # 12,180 — where did 214 go?

gdf.geometry.area.sum()                         # 4_812_003.2 — plausible, and wrong

Two hundred and fourteen rows silently matched nothing. The area total is wrong by an amount nobody can quantify. And dropna() reported that it had cleaned the data.

The problem is that "broken geometry" is not one condition. It is four different conditions, each detected by a different check, each with a different consequence, and only one of them is caught by dropna().

Quick answer

Four states, four checks, four fixes:

State Test What it does downstream Fix
Null — no geometry object .geometry.isna() dropped by most ops, raises in some drop the row, or repair from attributes
Empty — an object with no coordinates .geometry.is_empty matches nothing, area 0, not caught by isna drop, or investigate the source
Missing — valid geometry, absent attributes .isna() on other columns joins produce nulls, groupbys skip fill, or drop by rule
Invalid — self-intersecting or malformed ~.is_valid predicates unreliable, ops may raise make_valid() 
def geometry_health(gdf):
    return {
        "rows":    len(gdf),
        "null":    int(gdf.geometry.isna().sum()),
        "empty":   int(gdf.geometry.is_empty.sum()),
        "invalid": int((~gdf.is_valid).sum()),
        "no_crs":  gdf.crs is None,
    }

print(geometry_health(gdf))
# {'rows': 12400, 'null': 6, 'empty': 214, 'invalid': 31, 'no_crs': False}

The 214 empties are the answer to the missing rows. They are not null, so dropna() left them; they have no coordinates, so they match nothing.

# the check that actually cleans
gdf = gdf[gdf.geometry.notna() & ~gdf.geometry.is_empty]
gdf["geometry"] = gdf.geometry.make_valid()

The four states side by side

Table of four geometry states against the checks that detect them and their downstream effects.
Only the first row is caught by dropna().

Step-by-step solution

Vertical steps from counting states through dropping, repairing and verifying.
Count first. A cleaning step that does not report what it removed is not auditable.

1. Null geometry — the row has no shape at all 

gdf.geometry.isna().sum()      # 6

In GeoPandas a null geometry is None, stored in the geometry column like any missing value. It happens when a source row had no geometry field, when a join brought in unmatched rows, or when a transformation failed for a single feature.

# what produced them?
gdf[gdf.geometry.isna()][["parcel_id", "ward", "surveyed"]]

If the attributes are intact, the row is a real record with a lost shape — worth investigating rather than deleting. If the whole row is empty, it is padding from the source file.

gdf = gdf[gdf.geometry.notna()]

2. Empty geometry — a shape with nothing in it 

from shapely.geometry import Polygon
empty = Polygon()

empty is None          # False — it is a real object
empty.is_empty         # True
empty.is_valid         # True  (!)
empty.area             # 0.0
empty.geom_type        # 'Polygon'

An empty geometry passes every check people usually run. It is not null, it is valid, it has a geometry type, and it writes to a file without complaint. It also intersects nothing, contains nothing, and contributes zero to every sum.

Empties are produced by operations far more often than they are read from files:

a.intersection(b)      # empty when they do not overlap
gdf.clip(boundary)     # empty for features fully outside
gdf.buffer(-50)        # empty when the shape is thinner than 100 m

That last one is the common trap: a negative buffer on narrow parcels silently empties them.

before = len(gdf)
gdf = gdf[~gdf.geometry.is_empty]
print(f"dropped {before - len(gdf)} empty geometries")

3. Missing attributes — the geometry is fine, the data is not 

gdf.isna().sum()
# parcel_id      0
# ward         842
# surveyed    3011
# geometry       6

Attribute nulls do not break spatial operations, so they survive every geometry check and then quietly change results:

gdf.groupby("ward")["area_m2"].sum()   # the 842 null wards vanish from the result

groupby drops null keys by default. The total no longer reconciles with the layer total, and nothing said so.

# make the absence visible rather than silent
gdf["ward"] = gdf["ward"].fillna("UNKNOWN")
totals = gdf.groupby("ward")["area_m2"].sum()
assert totals.sum() == pytest.approx(gdf["area_m2"].sum())

See how to handle missing and null values in spatial datasets for the full treatment.

4. Invalid geometry — the shape breaks the rules 

(~gdf.is_valid).sum()                          # 31
gdf[~gdf.is_valid].geometry.apply(lambda g: explain_validity(g)).value_counts()
# Self-intersection[325113.2 674881.9]    28
# Ring Self-intersection[...]              3

Invalid does not mean unusable — it means predicates and overlays give unreliable answers, because the shape has no unambiguous interior. Some operations raise TopologyException; worse, some return a plausible wrong result.

gdf["geometry"] = gdf.geometry.make_valid()
assert gdf.is_valid.all()

make_valid can change the geometry type — a bowtie polygon becomes a MultiPolygon, a degenerate one becomes a LineString or a GeometryCollection. Check afterwards:

print(gdf.geom_type.value_counts())
gdf = gdf[gdf.geom_type.isin(["Polygon", "MultiPolygon"])]

See how to fix invalid geometries and buffer(0) does not fix my invalid geometry.

5. Do them in the right order

Order matters, because each step changes what the next one sees:

def clean_geometry(gdf, report=None):
    r = report if report is not None else {}
    r["in"] = len(gdf)

    gdf = gdf[gdf.geometry.notna()]                 # 1. null — nothing to repair
    r["dropped_null"] = r["in"] - len(gdf)

    n = len(gdf)
    gdf = gdf[~gdf.geometry.is_empty]               # 2. empty — before validity
    r["dropped_empty"] = n - len(gdf)

    bad = (~gdf.is_valid).sum()                     # 3. invalid — repair, do not drop
    if bad:
        gdf = gdf.assign(geometry=gdf.geometry.make_valid())
        r["repaired"] = int(bad)

    n = len(gdf)                                    # 4. make_valid can produce empties
    gdf = gdf[~gdf.geometry.is_empty]
    r["empty_after_repair"] = n - len(gdf)

    r["out"] = len(gdf)
    return gdf, r

Step 4 is the one people miss. make_valid on a degenerate shape can return an empty geometry, so the empty check has to run again afterwards.

Code examples

Example 1: a health report you can put in a log 

import geopandas as gpd
from shapely.validation import explain_validity

def geometry_report(gdf, name="layer"):
    invalid = gdf[~gdf.is_valid]
    reasons = (
        invalid.geometry.apply(lambda g: explain_validity(g).split("[")[0])
        .value_counts().to_dict() if len(invalid) else {}
    )
    return {
        "layer": name,
        "rows": len(gdf),
        "crs": str(gdf.crs),
        "null": int(gdf.geometry.isna().sum()),
        "empty": int(gdf.geometry.is_empty.sum()),
        "invalid": len(invalid),
        "invalid_reasons": reasons,
        "types": gdf.geom_type.value_counts().to_dict(),
    }

print(geometry_report(gdf, "parcels"))
# {'layer': 'parcels', 'rows': 12400, 'crs': 'EPSG:27700', 'null': 6,
#  'empty': 214, 'invalid': 31,
#  'invalid_reasons': {'Self-intersection': 28, 'Ring Self-intersection': 3},
#  'types': {'Polygon': 12103, 'MultiPolygon': 297}}

One dict, JSON-serialisable, worth writing to the run log of every job. When a total looks wrong three months later, this is the record that says whether the input was already broken.

Example 2: catching empties the moment an operation creates them 

def clip_checked(gdf, boundary):
    out = gdf.clip(boundary)
    empties = out.geometry.is_empty.sum()
    if empties:
        # clip legitimately empties features outside the boundary — the
        # question is whether you expected that many
        print(f"clip produced {empties} empty geometries of {len(out)}")
        out = out[~out.geometry.is_empty]
    return out

Operations that routinely produce empties: clip, intersection, difference, negative buffer, and simplify with an aggressive tolerance.

Example 3: asserting the invariant instead of hoping 

def assert_usable(gdf, name="layer"):
    problems = []
    if gdf.geometry.isna().any():
        problems.append(f"{gdf.geometry.isna().sum()} null geometries")
    if gdf.geometry.is_empty.any():
        problems.append(f"{gdf.geometry.is_empty.sum()} empty geometries")
    if not gdf.is_valid.all():
        problems.append(f"{(~gdf.is_valid).sum()} invalid geometries")
    if gdf.crs is None:
        problems.append("no CRS")
    if problems:
        raise ValueError(f"{name} is not usable: " + "; ".join(problems))

Call it after loading and after every step that can produce empties. An exception at the point of damage beats a wrong number at the end.

Explanation

Flow showing operations that create empty geometries and the checks that catch them.
Empties are usually created by your own pipeline, not read from the file.

The reason these four states are so easy to confuse is that they live at different layers of the stack.

Null is a pandas concept. The geometry column is a column like any other, and pandas allows missing values in it. dropna(), fillna() and isna() all work — and only see this layer.

Empty is a Shapely and OGC concept. Polygon() with no coordinates is a well-defined object in the Simple Features model: it has a type, it is valid, and its point set is the empty set. Everything follows logically from that — it intersects nothing because there is nothing to intersect, its area is zero because it encloses nothing. Pandas has no opinion about it, because to pandas it is just an object in a cell.

Invalid is a GEOS concept, defined by the OGC validity rules: rings must not self-intersect, interior rings must be inside the exterior, and so on. An invalid geometry is a real object with real coordinates that happens to describe something contradictory — like a polygon whose boundary crosses itself, so "inside" is undefined in the crossing region.

Missing attributes are a plain data concept and have nothing to do with geometry at all — but they change the same totals, and they are usually discovered at the same moment, which is why they belong in the same mental checklist.

The practical consequence: no single check covers all four, and the most common cleaning line in Python GIS — dropna(subset=["geometry"]) — covers exactly one. The 214 empty rows in the opening example were created by a clip earlier in the same script, survived the cleaning step that was supposed to catch them, and quietly subtracted an unknown area from the total.

Edge cases or notes

  • is_empty on a null geometry raises or returns False depending on version. Filter nulls first, then empties.
  • is_valid is True for empty geometries. Emptiness and validity are independent.
  • A GeometryCollection from make_valid may contain a mix of types. ST_CollectionExtract's equivalent in Python is filtering the parts you want.
  • Shapefiles cannot store null geometry — they write an empty shape instead, which is why round-tripping through shapefile silently converts nulls into empties.
  • GeoJSON writes null geometry as "geometry": null and reads it back as null, so it round-trips correctly.
  • gdf.explode() drops empty parts silently, which can change row counts in a way that looks like data loss.
  • An all-empty layer still has a CRS and a schema, so it writes a valid file with zero usable features. Assert on len() after filtering.
  • unary_union of a set containing empties works fine — they contribute nothing, which is the mathematically correct behaviour and occasionally a surprise.

FAQ

Why does dropna() not remove empty geometries?

Because an empty geometry is not missing. It is a real Shapely object with a type and a validity state; its point set just happens to be empty. Use ~gdf.geometry.is_empty as a separate filter.

Is an empty geometry valid?

Yes. Polygon().is_valid is True. Validity asks whether the shape breaks the OGC rules; an empty shape breaks none of them.

What creates empty geometries?

Almost always an operation rather than a file: clip, intersection and difference on non-overlapping inputs, negative buffer on narrow shapes, and aggressive simplify.

Should I drop invalid geometries or repair them?

Repair with make_valid, then check the result. Dropping loses real features, and the underlying problem is usually a digitising artefact rather than a wrong record.

How do I know why a geometry is invalid?

from shapely.validation import explain_validity returns a reason and the coordinate where it occurs — useful for finding a systematic problem in one source.

Does an empty geometry break a spatial join?

Not with an error. It simply matches nothing, so the row disappears from an inner join and gets nulls in a left join. That silence is exactly the problem.

What is the minimum check before analysis?

notna(), ~is_empty, is_valid.all() and crs is not None. Four lines, and they cover every state on this page.