How to Clean and Normalise Attribute Columns in a GeoDataFrame

On this page

A GeoDataFrame is just a pandas DataFrame with one (or more) geometry columns bolted on. That means every messy-data problem you have ever hit in pandas β€” stray whitespace, numbers stored as text, five spellings of the same category β€” shows up in spatial data too. The good news: you can fix all of it with ordinary pandas operations, as long as you are careful to leave the geometry column alone.

This guide walks through cleaning the attribute (non-geometry) columns of a GeoDataFrame: tidying headers, normalising strings, fixing dtypes, mapping categories to a canonical set, and unifying missing-value tokens. The geometry stays untouched the whole way.

Problem statement

You loaded a GeoDataFrame and the attribute table is a mess. Typical symptoms:

  • Messy headers β€” column names like " Site Name ", Pop.2020, COUNTY, with leading/trailing spaces, mixed case, punctuation, or outright duplicates.
  • Numbers stored as text β€” a population column with dtype object because one cell contains "1,234" or "N/A", so you cannot do arithmetic or comparisons.
  • Mixed casing and whitespace in values β€” "Forest", "forest ", " FOREST" are treated as three different categories.
  • Inconsistent categories β€” "NY", "N.Y.", "New York", "new york" all meaning the same thing.
  • Mixed null tokens β€” missing values represented as "NA", "N/A", "", "null", "-" instead of a real NaN, so isna() misses them.

None of these are geometry problems. The geometry column is fine β€” it is the attribute columns that need work.

Quick answer

Identify the text columns, explicitly exclude the active geometry column via gdf.geometry.name, then tidy names, strip whitespace, unify null tokens, and coerce dtypes column by column. Work on a copy and assign back with bracket assignment so you never trip chained-assignment warnings. Geometry is never touched because it is never in the list of columns you transform.

Messy attribute columns before cleaning and tidy, correctly typed columns after. Before ' Site Name ' (object) POP_2020: '1,234' type: 'Park ' / 'park' missing: 'NA', '', '-' After site_name (string) pop_2020: 1234 (Int64) type: 'Park' missing: <NA> spatialworkflow.io
Cleaning standardises names, types, and missing-value tokens β€” the geometry is never touched.

The examples use a real, messy attribute table: the Food Standards Agency's hygiene register for Camden, 4,293 food businesses, as the FSA publishes it (set up under Code examples).

import geopandas as gpd
import pandas as pd

gdf = gpd.read_file("camden_fhrs.gpkg").copy()

# 1. Tidy column names: split camelCase, then slugify
gdf.columns = (
    gdf.columns.str.replace(r"(?<=[a-z0-9])(?=[A-Z])", "_", regex=True)
    .str.strip().str.lower().str.replace(r"[^\w]+", "_", regex=True)
)

# 2. Identify text columns, excluding the geometry column
geom_col = gdf.geometry.name
str_cols = [c for c in gdf.select_dtypes(include=["object", "string"]).columns if c != geom_col]

# 3. Strip whitespace and unify missing-value tokens
null_tokens = ["", "na", "n/a", "null", "none", "-"]
for col in str_cols:
    s = gdf[col].astype("string").str.strip().str.replace(r"\s+", " ", regex=True)
    gdf[col] = s.mask(s.str.lower().isin(null_tokens))

# 4. Coerce obvious numeric / date columns (errors -> NaN, never crash)
gdf["rating"] = pd.to_numeric(gdf["rating_value"], errors="coerce")
gdf["rating_date"] = pd.to_datetime(gdf["rating_date"], errors="coerce")

# 5. Confirm geometry survived
assert gdf.geometry.name == geom_col
print(gdf.crs, gdf.geometry.geom_type.value_counts().to_dict(), "no geometry:", gdf.geometry.isna().sum())
print(gdf[["business_name", "rating_value", "rating", "rating_date"]].head(3).to_string())
EPSG:4326 {'Point': 3832} no geometry: 461
        business_name        rating_value  rating rating_date
0             1 REBEL                   5       5  2026-03-18
1         1+1 Rougamo                   4       4  2023-02-08
2  100% NATURAL JUICE  AwaitingInspection    <NA>         NaT

Names, text, numbers, dates and categories, in order

The order of passes when normalising attribute columns. one pass at a time, in this order1Tidy the column namessnake_case2Strip whitespace, normalise case.str.strip()3Fix dtypes β€” numbers and datesto_numeric, to_datetime4Unify the missing-value tokens"", "NA", -999 β†’ NaN5Map categories to a fixed setreplace({...})6Confirm geometry is untouchedstill a GeoDataFrameDtype conversion after trimming: " 42 " becomes a number, "42 " may not spatialworkflow.io
The order is not arbitrary β€” each pass makes the next one possible.

Inspect columns and dtypes

Before changing anything, look at what you have. dtypes tells you which columns pandas thinks are text versus numbers or dates, and head() shows you the actual values. Note which column is the active geometry β€” you will protect it throughout.

gdf = gpd.read_file("camden_fhrs.gpkg")

print(gdf.shape)
print(gdf.dtypes.astype(str).to_dict())
print("geometry column:", gdf.geometry.name, "| CRS:", gdf.crs)
(4293, 24)
{'FHRSID': 'int64', 'LocalAuthorityBusinessID': 'str', 'BusinessName': 'str', 'BusinessType': 'str', 'BusinessTypeID': 'int64', 'AddressLine1': 'str', 'AddressLine2': 'str', 'AddressLine4': 'str', 'PostCode': 'str', 'RatingValue': 'str', 'RatingKey': 'str', 'RatingDate': 'str', 'LocalAuthorityCode': 'str', 'LocalAuthorityName': 'str', 'LocalAuthorityWebSite': 'str', 'LocalAuthorityEmailAddress': 'str', 'SchemeType': 'str', 'NewRatingPending': 'bool', 'Scores.Hygiene': 'float64', 'Scores.Structural': 'float64', 'Scores.ConfidenceInManagement': 'float64', 'Geocode.Longitude': 'str', 'Geocode.Latitude': 'str', 'geometry': 'geometry'}
geometry column: geometry | CRS: EPSG:4326

Anything that should be a number or a date but shows up as text is a candidate for coercion β€” here RatingValue, which mixes digits with AwaitingInspection and Exempt, and RatingDate. Anything textual that should stay text is a candidate for whitespace and casing cleanup.

With pandas 3, text columns have the dtype str, not object. select_dtypes(include="object") still picks them up for now, with a Pandas4Warning that this will stop in pandas 4 β€” so ask for include=["object", "string"] and the code works on both.

Tidy column names

Clean headers make every later step easier. Strip whitespace, lowercase, replace runs of non-word characters with a single underscore, and then deal with duplicates. Renaming columns never touches the geometry data β€” it only relabels the geometry column, and GeoPandas tracks it by name, so set the geometry again if you rename it.

import re

plain = gdf.columns.str.strip().str.lower().str.replace(r"[^\w]+", "_", regex=True).str.strip("_")
print("slugify only:   ", list(plain[:4]), "…")

# Split camelCase first, then slugify
gdf.columns = (
    gdf.columns
    .str.replace(r"(?<=[a-z0-9])(?=[A-Z])", "_", regex=True)
    .str.strip()
    .str.lower()
    .str.replace(r"[^\w]+", "_", regex=True)
    .str.strip("_")
)

# De-duplicate names: site, site -> site, site_1
def dedupe(names):
    seen = {}
    out = []
    for n in names:
        if n in seen:
            seen[n] += 1
            out.append(f"{n}_{seen[n]}")
        else:
            seen[n] = 0
            out.append(n)
    return out

gdf.columns = dedupe(gdf.columns)
print("camelCase split:", list(gdf.columns))
slugify only:    ['fhrsid', 'localauthoritybusinessid', 'businessname', 'businesstype'] …
camelCase split: ['fhrsid', 'local_authority_business_id', 'business_name', 'business_type', 'business_type_id', 'address_line1', 'address_line2', 'address_line4', 'post_code', 'rating_value', 'rating_key', 'rating_date', 'local_authority_code', 'local_authority_name', 'local_authority_web_site', 'local_authority_email_address', 'scheme_type', 'new_rating_pending', 'scores_hygiene', 'scores_structural', 'scores_confidence_in_management', 'geocode_longitude', 'geocode_latitude', 'geometry']

The FSA's names are camelCase, so a plain slugify turns BusinessName into businessname; splitting at each lower-to-upper boundary first gives business_name. The dotted Scores.Hygiene becomes scores_hygiene. geometry is untouched by all of this, and if a rename ever does change the geometry column's name, re-establish it with gdf = gdf.set_geometry("<new name>").

Strip whitespace and normalise casing

Target only the text columns, and explicitly drop the geometry column from that list. Use the nullable string dtype so empty results become <NA> rather than the literal float nan. Here we strip whitespace, collapse internal runs of spaces, and compare names case-insensitively.

geom_col = gdf.geometry.name
str_cols = [c for c in gdf.select_dtypes(include=["object", "string"]).columns if c != geom_col]

names = gdf["business_name"].astype("string")
print("names with doubled spaces:", names.str.contains("  ").sum())

for col in str_cols:
    s = gdf[col].astype("string").str.strip()
    s = s.str.replace(r"\s+", " ", regex=True)   # collapse internal whitespace
    gdf[col] = s

key = gdf["business_name"].str.casefold()
print(f"distinct names: {names.nunique():,} as written, {key.nunique():,} ignoring case and spacing")
variants = gdf.groupby(key)["business_name"].unique()
print(variants[variants.map(len) > 1].head(4).map(list).to_dict())
names with doubled spaces: 22
distinct names: 3,960 as written, 3,886 ignoring case and spacing
{'acai berry': ['ACAI BERRY', 'Acai Berry'], 'amorino': ['Amorino', 'AMORINO'], 'asian eatery': ['Asian Eatery', 'ASIAN EATERY'], 'b bagel': ['B Bagel', 'B BAGEL']}

Pick casing per column, and keep the original: here 3,960 distinct names become 3,886 once case and spacing are ignored, and whether AMORINO and Amorino are the same business is a question the casing cannot answer β€” group on the casefolded key, display the original. Upper case suits short codes such as postcodes; title case mangles names like McDonald's and B Bagel.

Fix numeric and date dtypes

The key tool is pd.to_numeric(..., errors="coerce"), which turns anything it cannot parse into NaN instead of raising. For dates, pd.to_datetime(..., errors="coerce") does the same.

print(gdf["rating_value"].value_counts().to_dict())

gdf["rating"] = pd.to_numeric(gdf["rating_value"], errors="coerce")
gdf["rating_date"] = pd.to_datetime(gdf["rating_date"], errors="coerce")
gdf["latitude"] = pd.to_numeric(gdf["geocode_latitude"], errors="coerce")

print(gdf[["rating", "rating_date", "latitude"]].dtypes.astype(str).to_dict())
print("not a number:", gdf["rating"].isna().sum(), "| no date:", gdf["rating_date"].isna().sum())
{'5': 1974, 'AwaitingInspection': 863, '4': 670, '3': 448, '2': 124, '1': 91, 'Exempt': 78, '0': 45}
{'rating': 'Int64', 'rating_date': 'datetime64[us]', 'latitude': 'Float64'}
not a number: 941 | no date: 941

After coercion, check how many values failed. Here 941 ratings are not numbers β€” and that is correct, not a parsing error: they are the 863 businesses awaiting inspection and the 78 exempt ones, which is why the text column is kept beside the numeric one. A spike you cannot explain like that tells you a parsing assumption was wrong.

Normalise categorical values

Normalising categorical values so the unmatched ones report themselves. the NaNs are the featurestrip + lowercasea comparable keymap() thedictionaryyour source of truthunmatched becomeNaNon purposereport, thenextendthe dictionary growsThe NaNs are the values your dictionary has not heard of yet, which is what you wanted to know spatialworkflow.io
Silently keeping an unmatched value is what lets "Woodland " survive as its own category.

Map every known variant to one canonical value. Build a dictionary (your source of truth), normalise the raw values to a comparable form first (strip + lowercase), then map. Use .map() with care β€” unmatched values become NaN, which is useful for spotting unknowns.

status_map = {
    "0": "Rated", "1": "Rated", "2": "Rated", "3": "Rated", "4": "Rated", "5": "Rated",
    "awaitinginspection": "Awaiting inspection",
    "exempt": "Exempt",
}

key = gdf["rating_value"].str.strip().str.lower()
gdf["rating_status"] = key.map(status_map)

unmatched = gdf.loc[gdf["rating_status"].isna() & key.notna()]
print(gdf["rating_status"].value_counts().to_dict(), "| unmapped:", sorted(key[unmatched.index].unique()))
{'Rated': 3352, 'Awaiting inspection': 863, 'Exempt': 78} | unmapped: []

The same approach standardises codes. UK postcodes should be the outward code, a space and a three-character inward code; the register has a few that are not:

pattern = r"^[A-Z]{1,2}\d[A-Z\d]? \d[A-Z]{2}$"
raw = gdf["post_code"].str.upper()
fixed = raw.str.replace(r"\s+", "", regex=True).str.replace(r"^(.+)(\d[A-Z]{2})$", r"\1 \2", regex=True)
bad_before = raw.notna() & ~raw.str.match(pattern).fillna(False)
bad_after = fixed.notna() & ~fixed.str.match(pattern).fillna(False)
print(f"malformed: {bad_before.sum()} before, {bad_after.sum()} after re-spacing;",
      "still wrong:", raw[bad_after].tolist(), "| missing:", gdf["post_code"].isna().sum())
gdf["post_code"] = fixed
malformed: 8 before, 2 after re-spacing; still wrong: ['W1T ATT', 'WC2'] | missing: 177

Re-spacing repairs the ones that only lost their space. What is left is not a formatting problem: WC2 is half a postcode and W1T ATT is not one at all, and no regular expression will recover them β€” they go on a list for someone who can check the address.

Unify missing-value tokens

Different exports use different strings for "missing". Replace them all with a real NaN so isna(), dropna(), and fillna() work. Do this across all string columns at once, comparing case-insensitively.

null_tokens = {"", "na", "n/a", "null", "none", "nan", "-", "--", "unknown"}

found = {}
for col in str_cols:
    s = gdf[col].astype("string").str.strip()
    hits = s.str.lower().isin(null_tokens)
    if hits.any():
        found[col] = int(hits.sum())
    gdf[col] = s.mask(hits)

real_nulls = gdf[str_cols].isna().sum()
print("null tokens found:", found or "none", "| text columns with real nulls:", int((real_nulls > 0).sum()))
null tokens found: none | text columns with real nulls: 7

mask(cond) replaces values where cond is True with NA. In the FSA's JSON there were none to replace β€” missing values arrive as real nulls β€” which is worth knowing too: run the check, and when it finds nothing, the pass costs nothing. CSV exports from spreadsheets are where these tokens usually come from.

Verify geometry is intact

You never put the geometry column in any transform list, so it should be untouched β€” but verify. Confirm the active geometry name, the CRS, the geometry types, and that nothing became invalid or empty.

assert gdf.geometry.name == geom_col, "geometry column changed!"
print("CRS:", gdf.crs)
print("types:", gdf.geometry.geom_type.value_counts().to_dict())
print("empty:", gdf.geometry.is_empty.sum(), "| invalid:", (~gdf.geometry.is_valid).sum(),
      "| missing:", gdf.geometry.isna().sum())
CRS: EPSG:4326
types: {'Point': 3832}
empty: 0 | invalid: 461 | missing: 461

The missing geometries are the businesses the FSA publishes without a location β€” 461 of them here. They were missing on the way in, and they are exactly as missing on the way out. Note that the invalid count includes them: is_valid is False for a missing geometry, so read the two numbers together before concluding that anything was damaged.

Cleaning the columns of Camden's food register

The data used on this page

The Food Standards Agency's food hygiene rating register for Camden (FSA authority code 506), from its open data, written to a GeoPackage as delivered: the FSA's own field names, every value as the JSON carries it, and a point for each business with a geocode. The register changes as the council updates it, so counts drift a little between runs.

from pathlib import Path

import geopandas as gpd
import pandas as pd
import requests
from shapely.geometry import Point

if not Path("camden_fhrs.gpkg").exists():
    # the FSA register file as saved on 9 October 2026 (updated daily); live:
    # ratings.food.gov.uk/api/open-data-files/FHRS506en-GB.json
    reply = requests.get("https://www.spatialworkflow.io/data/fhrs-506-2026-10-09.json", timeout=300)
    reply.raise_for_status()
    records = reply.json()["FHRSEstablishment"]["EstablishmentCollection"]
    raw = pd.json_normalize(records).drop(columns=["Scores", "Geocode", "RightToReply"], errors="ignore")
    lon, lat = pd.to_numeric(raw["Geocode.Longitude"]), pd.to_numeric(raw["Geocode.Latitude"])
    geometry = [Point(x, y) if pd.notna(x) and pd.notna(y) else None for x, y in zip(lon, lat)]
    gpd.GeoDataFrame(raw, geometry=geometry, crs=4326).to_file("camden_fhrs.gpkg")

delivered = gpd.read_file("camden_fhrs.gpkg")
print(f"{len(delivered):,} businesses, {delivered.shape[1]} columns:", list(delivered.columns)[:6], "…")
4,293 businesses, 24 columns: ['FHRSID', 'LocalAuthorityBusinessID', 'BusinessName', 'BusinessType', 'BusinessTypeID', 'AddressLine1'] …

Example 1 β€” One-liner header slugify, camelCase included

cols = delivered.columns.str.replace(r"(?<=[a-z0-9])(?=[A-Z])", "_", regex=True)
print(list(cols.str.strip().str.lower().str.replace(r"[^\w]+", "_", regex=True).str.strip("_"))[-8:])
['scheme_type', 'new_rating_pending', 'scores_hygiene', 'scores_structural', 'scores_confidence_in_management', 'geocode_longitude', 'geocode_latitude', 'geometry']

Example 2 β€” Strip and collapse whitespace on every text column except geometry

work = delivered.copy()
text = [c for c in work.select_dtypes(include=["object", "string"]).columns if c != work.geometry.name]
before = work[text].astype("string").apply(lambda s: s.str.contains(r"^\s|\s$|\s\s", regex=True).sum()).sum()

work[text] = work[text].apply(
    lambda s: s.astype("string").str.strip().str.replace(r"\s+", " ", regex=True)
)
print(f"{len(text)} text columns; {before} values had stray whitespace; geometry column still {work.geometry.name!r}")
17 text columns; 52 values had stray whitespace; geometry column still 'geometry'

Example 3 β€” Robust numeric coercion with cleanup

def to_number(s):
    s = s.astype("string").str.strip()
    s = s.str.replace(",", "", regex=False).str.replace("Β£", "", regex=False)
    return pd.to_numeric(s, errors="coerce")

for col in ["RatingValue", "Scores.Hygiene", "Geocode.Latitude"]:
    values = to_number(delivered[col])
    print(f"{col:18} {str(delivered[col].dtype):>7} -> {str(values.dtype):7} {values.isna().sum():>4} NaN")
RatingValue            str -> Int64    941 NaN
Scores.Hygiene     float64 -> Float64  944 NaN
Geocode.Latitude       str -> Float64  461 NaN

Example 4 β€” Category normalisation with an unmatched report

type_map = {"restaurant/cafe/canteen": "Food service", "takeaway/sandwich shop": "Food service",
            "pub/bar/nightclub": "Food service", "retailers - other": "Retail",
            "retailers - supermarkets/hypermarkets": "Retail"}

key = delivered["BusinessType"].str.strip().str.lower()
mapped = key.map(type_map)
print("unmapped:", sorted(key[mapped.isna() & key.notna()].unique()))
print(mapped.value_counts().to_dict())
unmapped: ['distributors/transporters', 'hospitals/childcare/caring premises', 'hotel/bed & breakfast/guest house', 'importers/exporters', 'manufacturers/packers', 'mobile caterer', 'other catering premises', 'school/college/university']
{'Food service': 2744, 'Retail': 1075}

The unmatched report is the point: eight business types the dictionary did not know about, listed instead of silently kept. Decide for each, then extend the map.

Example 5 β€” Unify null tokens across all attribute columns

null_tokens = {"", "na", "n/a", "null", "none", "nan", "-", "--", "unknown"}

for col in text:
    s = work[col].astype("string").str.strip()
    work[col] = s.mask(s.str.lower().isin(null_tokens))
print(work[text].isna().sum().loc[lambda n: n > 0].to_dict())
{'AddressLine1': 2972, 'AddressLine2': 178, 'AddressLine4': 479, 'PostCode': 177, 'RatingDate': 941, 'Geocode.Longitude': 461, 'Geocode.Latitude': 461}

Why columns arrive as object dtype

Why columns end up as object dtype. pandas picks one dtype per column. When a column contains any value it cannot store as a number β€” an empty cell read as "", a stray "N/A", a thousands separator like "1,234" β€” it falls back to a text dtype: the catch-all object in pandas 2, the dedicated str in pandas 3, which is what the FSA's RatingValue arrived as. So a single bad cell among a million good ones is enough to make the whole column text. This is why arithmetic silently fails or string-concatenates instead of adding.

The many ways a missing value can be encoded in a delivered dataset. all of these mean "missing"as delivereddtype it forcesconvert to""objectNaN" "objectNaN"NA" / "N/A"objectNaN"NULL" / "None"objectNaN-999 / -9999int64NaN0 (sometimes)int64ask the supplierA single "NA" string keeps a whole numeric column as object dtypeUnify these first, or every later numeric operation silently fails spatialworkflow.io
The last row is a judgement call β€” zero is often a real measurement.

Why one bad cell downgrades a column. Dtypes are column-wide, not per-cell. There is no "mostly integer" dtype; the column is either fully numeric or it is text. pd.to_numeric(errors="coerce") is the cure: it parses what it can and turns the rest into NaN, after which the column becomes a proper float64 (or nullable Int64/Float64) and the bad cells are explicit missing values you can inspect and decide about.

Keeping the geometry column out of transforms. The active geometry column is a geometry dtype, not text, so select_dtypes(include=["object", "string"]) would not normally pick it up β€” but older files, mixed sources, or a column that lost its geometry dtype can surprise you. The safe habit is to always read gdf.geometry.name and exclude it explicitly from any list of columns you loop over. You never call .str or to_numeric on the geometry, so it cannot be corrupted. When you rename columns, GeoPandas keeps tracking geometry by name, so re-set it with set_geometry if the name changed.

Keeping the geometry and dtypes intact

Deriving the working column list so bulk operations never touch the geometry column. the one column that must never be cleanedcols = [c for c in gdf.columnsif c != gdf.geometry.name]gdf[cols] = gdf[cols].apply(clean)β‘ β‘‘β‘ Not the literal "geometry" β€” the column can be named anything, and thisreads the name the frame actually uses.β‘‘Casting geometry to text means re-parsing with GeoSeries.from_wkt, orreloading the file.select_dtypes("object") excludes geometry by design β€” exclude it explicitly anyway spatialworkflow.io
Every bulk operation in this article derives its column list this way.

Never transform the geometry column

Do not apply string or numeric operations to the geometry column, and do not include it when you bulk-process columns. Always derive your working list as [c for c in ... if c != gdf.geometry.name]. If you accidentally cast geometry to text, you will need to re-parse it with gpd.GeoSeries.from_wkt or reload the file.

select_dtypes("object") targets strings β€” but verify

select_dtypes is the right way to grab text columns, and it excludes the geometry dtype by design. Ask for include=["object", "string"]: in pandas 3 text columns are str, which "object" alone still matches only with a Pandas4Warning that it will stop doing so. Still, exclude gdf.geometry.name explicitly: a column read from some formats can arrive as generic object, and you do not want to assume.

Thousands separators and locale

"1,234.5" is one number in US/UK convention but "1.234,5" in much of Europe. pd.to_numeric does not localise. Strip the grouping separator yourself (str.replace(",", "")) for US-style data, or for European-style swap separators first: s.str.replace(".", "", regex=False).str.replace(",", ".", regex=False). Decide based on the source, not a guess.

Category mapping needs a source of truth

A mapping is only as good as the dictionary behind it. Keep the canonical set in one place (a dict, a small lookup table, or a config file), normalise raw values before mapping (strip + lowercase), and always report values that did not match so you can extend the map rather than silently dropping them to NaN.

Encoding on read

Garbled characters like é instead of Γ© are an encoding problem, not a cleaning problem β€” fix them at read time. Pass the right encoding: gpd.read_file("data.shp", encoding="utf-8") (or latin-1 for legacy shapefiles). A shapefile's .cpg sidecar declares its encoding; if it is wrong or missing, override it. Cleaning after the fact is far harder than reading correctly.

Chained assignment β€” use .loc and .copy()

Start from gdf = gpd.read_file(...).copy() if the GeoDataFrame is a slice of another, and assign with gdf[col] = ... or gdf.loc[mask, col] = ... rather than gdf[col][mask] = .... Chained indexing can write to a temporary copy and silently do nothing (and raises SettingWithCopyWarning). On pandas 3.x with Copy-on-Write, chained assignment never works, so the .loc habit is mandatory.

Preserve nullable dtypes deliberately

After coercion you may want Int64 (nullable integer) instead of float64 so counts stay integers despite NaNs: gdf["rating"] = gdf["rating"].astype("Int64"). The nullable string dtype likewise keeps <NA> distinct from the literal string "nan", which matters when you unify null tokens.

FAQ

Does cleaning attribute columns ever change the geometry?

No, as long as you never include the geometry column in your transforms. String and numeric operations apply only to the columns you name or select, and select_dtypes(include=["object", "string"]) does not return the geometry dtype. Always exclude gdf.geometry.name explicitly as a safety net, then verify CRS and geometry types afterward.

Why is my number column an object dtype?

Because at least one value cannot be stored as a number β€” commonly an empty string, a token like "N/A", or a thousands separator such as "1,234". pandas falls back to text for the whole column β€” object in pandas 2, str in pandas 3. Run pd.to_numeric(col, errors="coerce") after stripping separators to convert it and turn the unparseable cells into NaN.

How do I convert messy text into a real NaN?

Build a set of the missing-value tokens your data uses ("NA", "N/A", "", "null", "-", etc.), then use s.mask(s.str.lower().isin(tokens)) on the nullable string dtype. That replaces each matching value with <NA>, after which isna(), dropna(), and fillna() all work as expected.

What is the difference between object and string dtype?

object is pandas' generic container for any Python object, and was what text read from a file became until pandas 3. Pandas 3 reads text as str, a dedicated string dtype that still marks missing values with NaN; .astype("string") gives the nullable variant with a proper <NA>, which is what the cleaning code above uses before masking null tokens.

How do I rename columns without breaking the geometry?

Renaming is safe: GeoPandas tracks the active geometry by its column name, so reassigning gdf.columns only relabels it. If your cleanup renames the geometry column itself, re-establish it with gdf = gdf.set_geometry("new_name") so gdf.geometry keeps pointing at the right column.

Should I use Int64 or float64 for integer columns with missing values?

Use Int64 (capital I, the nullable integer dtype) when you need values to stay integers despite missing entries β€” for example ID codes or counts. Plain float64 will display 1.0 and admits floating-point fuzz. Coerce first with pd.to_numeric(..., errors="coerce"), then .astype("Int64").

How do I fix garbled characters like é?

That is an encoding mismatch, so fix it when reading rather than cleaning afterward. Pass the correct encoding to gpd.read_file(path, encoding="utf-8"), or latin-1 for older shapefiles. Check the shapefile's .cpg sidecar β€” if it declares the wrong encoding, override it explicitly on read.

Why does my assignment seem to do nothing?

You are probably hitting chained assignment, like gdf[col][mask] = value, which writes to a temporary copy. Use gdf.loc[mask, col] = value instead, and start from a .copy() if your GeoDataFrame is a slice of another. On pandas 3.x with Copy-on-Write, the .loc form is the only one that works.