How to Clean and Normalise Attribute Columns in a GeoDataFrame
On this page
A GeoDataFrame is just a pandas DataFrame with one (or more) geometry columns bolted on. That means every messy-data problem you have ever hit in pandas β stray whitespace, numbers stored as text, five spellings of the same category β shows up in spatial data too. The good news: you can fix all of it with ordinary pandas operations, as long as you are careful to leave the geometry column alone.
This guide walks through cleaning the attribute (non-geometry) columns of a GeoDataFrame: tidying headers, normalising strings, fixing dtypes, mapping categories to a canonical set, and unifying missing-value tokens. The geometry stays untouched the whole way.
Problem statement
You loaded a GeoDataFrame and the attribute table is a mess. Typical symptoms:
- Messy headers β column names like
" Site Name ",Pop.2020,COUNTY, with leading/trailing spaces, mixed case, punctuation, or outright duplicates. - Numbers stored as text β a
populationcolumn with dtypeobjectbecause one cell contains"1,234"or"N/A", so you cannot do arithmetic or comparisons. - Mixed casing and whitespace in values β
"Forest","forest "," FOREST"are treated as three different categories. - Inconsistent categories β
"NY","N.Y.","New York","new york"all meaning the same thing. - Mixed null tokens β missing values represented as
"NA","N/A","","null","-"instead of a realNaN, soisna()misses them.
None of these are geometry problems. The geometry column is fine β it is the attribute columns that need work.
Quick answer
Identify the text columns, explicitly exclude the active geometry column via gdf.geometry.name, then tidy names, strip whitespace, unify null tokens, and coerce dtypes column by column. Work on a copy and assign back with bracket assignment so you never trip chained-assignment warnings. Geometry is never touched because it is never in the list of columns you transform.
The examples use a real, messy attribute table: the Food Standards Agency's hygiene register for Camden, 4,293 food businesses, as the FSA publishes it (set up under Code examples).
import geopandas as gpd
import pandas as pd
gdf = gpd.read_file("camden_fhrs.gpkg").copy()
# 1. Tidy column names: split camelCase, then slugify
gdf.columns = (
gdf.columns.str.replace(r"(?<=[a-z0-9])(?=[A-Z])", "_", regex=True)
.str.strip().str.lower().str.replace(r"[^\w]+", "_", regex=True)
)
# 2. Identify text columns, excluding the geometry column
geom_col = gdf.geometry.name
str_cols = [c for c in gdf.select_dtypes(include=["object", "string"]).columns if c != geom_col]
# 3. Strip whitespace and unify missing-value tokens
null_tokens = ["", "na", "n/a", "null", "none", "-"]
for col in str_cols:
s = gdf[col].astype("string").str.strip().str.replace(r"\s+", " ", regex=True)
gdf[col] = s.mask(s.str.lower().isin(null_tokens))
# 4. Coerce obvious numeric / date columns (errors -> NaN, never crash)
gdf["rating"] = pd.to_numeric(gdf["rating_value"], errors="coerce")
gdf["rating_date"] = pd.to_datetime(gdf["rating_date"], errors="coerce")
# 5. Confirm geometry survived
assert gdf.geometry.name == geom_col
print(gdf.crs, gdf.geometry.geom_type.value_counts().to_dict(), "no geometry:", gdf.geometry.isna().sum())
print(gdf[["business_name", "rating_value", "rating", "rating_date"]].head(3).to_string())
EPSG:4326 {'Point': 3832} no geometry: 461
business_name rating_value rating rating_date
0 1 REBEL 5 5 2026-03-18
1 1+1 Rougamo 4 4 2023-02-08
2 100% NATURAL JUICE AwaitingInspection <NA> NaT
Names, text, numbers, dates and categories, in order
Inspect columns and dtypes
Before changing anything, look at what you have. dtypes tells you which columns pandas thinks are text versus numbers or dates, and head() shows you the actual values. Note which column is the active geometry β you will protect it throughout.
gdf = gpd.read_file("camden_fhrs.gpkg")
print(gdf.shape)
print(gdf.dtypes.astype(str).to_dict())
print("geometry column:", gdf.geometry.name, "| CRS:", gdf.crs)
(4293, 24)
{'FHRSID': 'int64', 'LocalAuthorityBusinessID': 'str', 'BusinessName': 'str', 'BusinessType': 'str', 'BusinessTypeID': 'int64', 'AddressLine1': 'str', 'AddressLine2': 'str', 'AddressLine4': 'str', 'PostCode': 'str', 'RatingValue': 'str', 'RatingKey': 'str', 'RatingDate': 'str', 'LocalAuthorityCode': 'str', 'LocalAuthorityName': 'str', 'LocalAuthorityWebSite': 'str', 'LocalAuthorityEmailAddress': 'str', 'SchemeType': 'str', 'NewRatingPending': 'bool', 'Scores.Hygiene': 'float64', 'Scores.Structural': 'float64', 'Scores.ConfidenceInManagement': 'float64', 'Geocode.Longitude': 'str', 'Geocode.Latitude': 'str', 'geometry': 'geometry'}
geometry column: geometry | CRS: EPSG:4326
Anything that should be a number or a date but shows up as text is a candidate for coercion β here RatingValue, which mixes digits with AwaitingInspection and Exempt, and RatingDate. Anything textual that should stay text is a candidate for whitespace and casing cleanup.
With pandas 3, text columns have the dtype str, not object. select_dtypes(include="object") still picks them up for now, with a Pandas4Warning that this will stop in pandas 4 β so ask for include=["object", "string"] and the code works on both.
Tidy column names
Clean headers make every later step easier. Strip whitespace, lowercase, replace runs of non-word characters with a single underscore, and then deal with duplicates. Renaming columns never touches the geometry data β it only relabels the geometry column, and GeoPandas tracks it by name, so set the geometry again if you rename it.
import re
plain = gdf.columns.str.strip().str.lower().str.replace(r"[^\w]+", "_", regex=True).str.strip("_")
print("slugify only: ", list(plain[:4]), "β¦")
# Split camelCase first, then slugify
gdf.columns = (
gdf.columns
.str.replace(r"(?<=[a-z0-9])(?=[A-Z])", "_", regex=True)
.str.strip()
.str.lower()
.str.replace(r"[^\w]+", "_", regex=True)
.str.strip("_")
)
# De-duplicate names: site, site -> site, site_1
def dedupe(names):
seen = {}
out = []
for n in names:
if n in seen:
seen[n] += 1
out.append(f"{n}_{seen[n]}")
else:
seen[n] = 0
out.append(n)
return out
gdf.columns = dedupe(gdf.columns)
print("camelCase split:", list(gdf.columns))
slugify only: ['fhrsid', 'localauthoritybusinessid', 'businessname', 'businesstype'] β¦
camelCase split: ['fhrsid', 'local_authority_business_id', 'business_name', 'business_type', 'business_type_id', 'address_line1', 'address_line2', 'address_line4', 'post_code', 'rating_value', 'rating_key', 'rating_date', 'local_authority_code', 'local_authority_name', 'local_authority_web_site', 'local_authority_email_address', 'scheme_type', 'new_rating_pending', 'scores_hygiene', 'scores_structural', 'scores_confidence_in_management', 'geocode_longitude', 'geocode_latitude', 'geometry']
The FSA's names are camelCase, so a plain slugify turns BusinessName into businessname; splitting at each lower-to-upper boundary first gives business_name. The dotted Scores.Hygiene becomes scores_hygiene. geometry is untouched by all of this, and if a rename ever does change the geometry column's name, re-establish it with gdf = gdf.set_geometry("<new name>").
Strip whitespace and normalise casing
Target only the text columns, and explicitly drop the geometry column from that list. Use the nullable string dtype so empty results become <NA> rather than the literal float nan. Here we strip whitespace, collapse internal runs of spaces, and compare names case-insensitively.
geom_col = gdf.geometry.name
str_cols = [c for c in gdf.select_dtypes(include=["object", "string"]).columns if c != geom_col]
names = gdf["business_name"].astype("string")
print("names with doubled spaces:", names.str.contains(" ").sum())
for col in str_cols:
s = gdf[col].astype("string").str.strip()
s = s.str.replace(r"\s+", " ", regex=True) # collapse internal whitespace
gdf[col] = s
key = gdf["business_name"].str.casefold()
print(f"distinct names: {names.nunique():,} as written, {key.nunique():,} ignoring case and spacing")
variants = gdf.groupby(key)["business_name"].unique()
print(variants[variants.map(len) > 1].head(4).map(list).to_dict())
names with doubled spaces: 22
distinct names: 3,960 as written, 3,886 ignoring case and spacing
{'acai berry': ['ACAI BERRY', 'Acai Berry'], 'amorino': ['Amorino', 'AMORINO'], 'asian eatery': ['Asian Eatery', 'ASIAN EATERY'], 'b bagel': ['B Bagel', 'B BAGEL']}
Pick casing per column, and keep the original: here 3,960 distinct names become 3,886 once case and spacing are ignored, and whether AMORINO and Amorino are the same business is a question the casing cannot answer β group on the casefolded key, display the original. Upper case suits short codes such as postcodes; title case mangles names like McDonald's and B Bagel.
Fix numeric and date dtypes
The key tool is pd.to_numeric(..., errors="coerce"), which turns anything it cannot parse into NaN instead of raising. For dates, pd.to_datetime(..., errors="coerce") does the same.
print(gdf["rating_value"].value_counts().to_dict())
gdf["rating"] = pd.to_numeric(gdf["rating_value"], errors="coerce")
gdf["rating_date"] = pd.to_datetime(gdf["rating_date"], errors="coerce")
gdf["latitude"] = pd.to_numeric(gdf["geocode_latitude"], errors="coerce")
print(gdf[["rating", "rating_date", "latitude"]].dtypes.astype(str).to_dict())
print("not a number:", gdf["rating"].isna().sum(), "| no date:", gdf["rating_date"].isna().sum())
{'5': 1974, 'AwaitingInspection': 863, '4': 670, '3': 448, '2': 124, '1': 91, 'Exempt': 78, '0': 45}
{'rating': 'Int64', 'rating_date': 'datetime64[us]', 'latitude': 'Float64'}
not a number: 941 | no date: 941
After coercion, check how many values failed. Here 941 ratings are not numbers β and that is correct, not a parsing error: they are the 863 businesses awaiting inspection and the 78 exempt ones, which is why the text column is kept beside the numeric one. A spike you cannot explain like that tells you a parsing assumption was wrong.
Normalise categorical values
Map every known variant to one canonical value. Build a dictionary (your source of truth), normalise the raw values to a comparable form first (strip + lowercase), then map. Use .map() with care β unmatched values become NaN, which is useful for spotting unknowns.
status_map = {
"0": "Rated", "1": "Rated", "2": "Rated", "3": "Rated", "4": "Rated", "5": "Rated",
"awaitinginspection": "Awaiting inspection",
"exempt": "Exempt",
}
key = gdf["rating_value"].str.strip().str.lower()
gdf["rating_status"] = key.map(status_map)
unmatched = gdf.loc[gdf["rating_status"].isna() & key.notna()]
print(gdf["rating_status"].value_counts().to_dict(), "| unmapped:", sorted(key[unmatched.index].unique()))
{'Rated': 3352, 'Awaiting inspection': 863, 'Exempt': 78} | unmapped: []
The same approach standardises codes. UK postcodes should be the outward code, a space and a three-character inward code; the register has a few that are not:
pattern = r"^[A-Z]{1,2}\d[A-Z\d]? \d[A-Z]{2}$"
raw = gdf["post_code"].str.upper()
fixed = raw.str.replace(r"\s+", "", regex=True).str.replace(r"^(.+)(\d[A-Z]{2})$", r"\1 \2", regex=True)
bad_before = raw.notna() & ~raw.str.match(pattern).fillna(False)
bad_after = fixed.notna() & ~fixed.str.match(pattern).fillna(False)
print(f"malformed: {bad_before.sum()} before, {bad_after.sum()} after re-spacing;",
"still wrong:", raw[bad_after].tolist(), "| missing:", gdf["post_code"].isna().sum())
gdf["post_code"] = fixed
malformed: 8 before, 2 after re-spacing; still wrong: ['W1T ATT', 'WC2'] | missing: 177
Re-spacing repairs the ones that only lost their space. What is left is not a formatting problem: WC2 is half a postcode and W1T ATT is not one at all, and no regular expression will recover them β they go on a list for someone who can check the address.
Unify missing-value tokens
Different exports use different strings for "missing". Replace them all with a real NaN so isna(), dropna(), and fillna() work. Do this across all string columns at once, comparing case-insensitively.
null_tokens = {"", "na", "n/a", "null", "none", "nan", "-", "--", "unknown"}
found = {}
for col in str_cols:
s = gdf[col].astype("string").str.strip()
hits = s.str.lower().isin(null_tokens)
if hits.any():
found[col] = int(hits.sum())
gdf[col] = s.mask(hits)
real_nulls = gdf[str_cols].isna().sum()
print("null tokens found:", found or "none", "| text columns with real nulls:", int((real_nulls > 0).sum()))
null tokens found: none | text columns with real nulls: 7
mask(cond) replaces values where cond is True with NA. In the FSA's JSON there were none to replace β missing values arrive as real nulls β which is worth knowing too: run the check, and when it finds nothing, the pass costs nothing. CSV exports from spreadsheets are where these tokens usually come from.
Verify geometry is intact
You never put the geometry column in any transform list, so it should be untouched β but verify. Confirm the active geometry name, the CRS, the geometry types, and that nothing became invalid or empty.
assert gdf.geometry.name == geom_col, "geometry column changed!"
print("CRS:", gdf.crs)
print("types:", gdf.geometry.geom_type.value_counts().to_dict())
print("empty:", gdf.geometry.is_empty.sum(), "| invalid:", (~gdf.geometry.is_valid).sum(),
"| missing:", gdf.geometry.isna().sum())
CRS: EPSG:4326
types: {'Point': 3832}
empty: 0 | invalid: 461 | missing: 461
The missing geometries are the businesses the FSA publishes without a location β 461 of them here. They were missing on the way in, and they are exactly as missing on the way out. Note that the invalid count includes them: is_valid is False for a missing geometry, so read the two numbers together before concluding that anything was damaged.
Cleaning the columns of Camden's food register
The data used on this page
The Food Standards Agency's food hygiene rating register for Camden (FSA authority code 506), from its open data, written to a GeoPackage as delivered: the FSA's own field names, every value as the JSON carries it, and a point for each business with a geocode. The register changes as the council updates it, so counts drift a little between runs.
from pathlib import Path
import geopandas as gpd
import pandas as pd
import requests
from shapely.geometry import Point
if not Path("camden_fhrs.gpkg").exists():
# the FSA register file as saved on 9 October 2026 (updated daily); live:
# ratings.food.gov.uk/api/open-data-files/FHRS506en-GB.json
reply = requests.get("https://www.spatialworkflow.io/data/fhrs-506-2026-10-09.json", timeout=300)
reply.raise_for_status()
records = reply.json()["FHRSEstablishment"]["EstablishmentCollection"]
raw = pd.json_normalize(records).drop(columns=["Scores", "Geocode", "RightToReply"], errors="ignore")
lon, lat = pd.to_numeric(raw["Geocode.Longitude"]), pd.to_numeric(raw["Geocode.Latitude"])
geometry = [Point(x, y) if pd.notna(x) and pd.notna(y) else None for x, y in zip(lon, lat)]
gpd.GeoDataFrame(raw, geometry=geometry, crs=4326).to_file("camden_fhrs.gpkg")
delivered = gpd.read_file("camden_fhrs.gpkg")
print(f"{len(delivered):,} businesses, {delivered.shape[1]} columns:", list(delivered.columns)[:6], "β¦")
4,293 businesses, 24 columns: ['FHRSID', 'LocalAuthorityBusinessID', 'BusinessName', 'BusinessType', 'BusinessTypeID', 'AddressLine1'] β¦
Example 1 β One-liner header slugify, camelCase included
cols = delivered.columns.str.replace(r"(?<=[a-z0-9])(?=[A-Z])", "_", regex=True)
print(list(cols.str.strip().str.lower().str.replace(r"[^\w]+", "_", regex=True).str.strip("_"))[-8:])
['scheme_type', 'new_rating_pending', 'scores_hygiene', 'scores_structural', 'scores_confidence_in_management', 'geocode_longitude', 'geocode_latitude', 'geometry']
Example 2 β Strip and collapse whitespace on every text column except geometry
work = delivered.copy()
text = [c for c in work.select_dtypes(include=["object", "string"]).columns if c != work.geometry.name]
before = work[text].astype("string").apply(lambda s: s.str.contains(r"^\s|\s$|\s\s", regex=True).sum()).sum()
work[text] = work[text].apply(
lambda s: s.astype("string").str.strip().str.replace(r"\s+", " ", regex=True)
)
print(f"{len(text)} text columns; {before} values had stray whitespace; geometry column still {work.geometry.name!r}")
17 text columns; 52 values had stray whitespace; geometry column still 'geometry'
Example 3 β Robust numeric coercion with cleanup
def to_number(s):
s = s.astype("string").str.strip()
s = s.str.replace(",", "", regex=False).str.replace("Β£", "", regex=False)
return pd.to_numeric(s, errors="coerce")
for col in ["RatingValue", "Scores.Hygiene", "Geocode.Latitude"]:
values = to_number(delivered[col])
print(f"{col:18} {str(delivered[col].dtype):>7} -> {str(values.dtype):7} {values.isna().sum():>4} NaN")
RatingValue str -> Int64 941 NaN
Scores.Hygiene float64 -> Float64 944 NaN
Geocode.Latitude str -> Float64 461 NaN
Example 4 β Category normalisation with an unmatched report
type_map = {"restaurant/cafe/canteen": "Food service", "takeaway/sandwich shop": "Food service",
"pub/bar/nightclub": "Food service", "retailers - other": "Retail",
"retailers - supermarkets/hypermarkets": "Retail"}
key = delivered["BusinessType"].str.strip().str.lower()
mapped = key.map(type_map)
print("unmapped:", sorted(key[mapped.isna() & key.notna()].unique()))
print(mapped.value_counts().to_dict())
unmapped: ['distributors/transporters', 'hospitals/childcare/caring premises', 'hotel/bed & breakfast/guest house', 'importers/exporters', 'manufacturers/packers', 'mobile caterer', 'other catering premises', 'school/college/university']
{'Food service': 2744, 'Retail': 1075}
The unmatched report is the point: eight business types the dictionary did not know about, listed instead of silently kept. Decide for each, then extend the map.
Example 5 β Unify null tokens across all attribute columns
null_tokens = {"", "na", "n/a", "null", "none", "nan", "-", "--", "unknown"}
for col in text:
s = work[col].astype("string").str.strip()
work[col] = s.mask(s.str.lower().isin(null_tokens))
print(work[text].isna().sum().loc[lambda n: n > 0].to_dict())
{'AddressLine1': 2972, 'AddressLine2': 178, 'AddressLine4': 479, 'PostCode': 177, 'RatingDate': 941, 'Geocode.Longitude': 461, 'Geocode.Latitude': 461}
Why columns arrive as object dtype
Why columns end up as object dtype. pandas picks one dtype per column. When a column contains any value it cannot store as a number β an empty cell read as "", a stray "N/A", a thousands separator like "1,234" β it falls back to a text dtype: the catch-all object in pandas 2, the dedicated str in pandas 3, which is what the FSA's RatingValue arrived as. So a single bad cell among a million good ones is enough to make the whole column text. This is why arithmetic silently fails or string-concatenates instead of adding.
Why one bad cell downgrades a column. Dtypes are column-wide, not per-cell. There is no "mostly integer" dtype; the column is either fully numeric or it is text. pd.to_numeric(errors="coerce") is the cure: it parses what it can and turns the rest into NaN, after which the column becomes a proper float64 (or nullable Int64/Float64) and the bad cells are explicit missing values you can inspect and decide about.
Keeping the geometry column out of transforms. The active geometry column is a geometry dtype, not text, so select_dtypes(include=["object", "string"]) would not normally pick it up β but older files, mixed sources, or a column that lost its geometry dtype can surprise you. The safe habit is to always read gdf.geometry.name and exclude it explicitly from any list of columns you loop over. You never call .str or to_numeric on the geometry, so it cannot be corrupted. When you rename columns, GeoPandas keeps tracking geometry by name, so re-set it with set_geometry if the name changed.
Keeping the geometry and dtypes intact
Never transform the geometry column
Do not apply string or numeric operations to the geometry column, and do not include it when you bulk-process columns. Always derive your working list as [c for c in ... if c != gdf.geometry.name]. If you accidentally cast geometry to text, you will need to re-parse it with gpd.GeoSeries.from_wkt or reload the file.
select_dtypes("object") targets strings β but verify
select_dtypes is the right way to grab text columns, and it excludes the geometry dtype by design. Ask for include=["object", "string"]: in pandas 3 text columns are str, which "object" alone still matches only with a Pandas4Warning that it will stop doing so. Still, exclude gdf.geometry.name explicitly: a column read from some formats can arrive as generic object, and you do not want to assume.
Thousands separators and locale
"1,234.5" is one number in US/UK convention but "1.234,5" in much of Europe. pd.to_numeric does not localise. Strip the grouping separator yourself (str.replace(",", "")) for US-style data, or for European-style swap separators first: s.str.replace(".", "", regex=False).str.replace(",", ".", regex=False). Decide based on the source, not a guess.
Category mapping needs a source of truth
A mapping is only as good as the dictionary behind it. Keep the canonical set in one place (a dict, a small lookup table, or a config file), normalise raw values before mapping (strip + lowercase), and always report values that did not match so you can extend the map rather than silently dropping them to NaN.
Encoding on read
Garbled characters like ΓΒ© instead of Γ© are an encoding problem, not a cleaning problem β fix them at read time. Pass the right encoding: gpd.read_file("data.shp", encoding="utf-8") (or latin-1 for legacy shapefiles). A shapefile's .cpg sidecar declares its encoding; if it is wrong or missing, override it. Cleaning after the fact is far harder than reading correctly.
Chained assignment β use .loc and .copy()
Start from gdf = gpd.read_file(...).copy() if the GeoDataFrame is a slice of another, and assign with gdf[col] = ... or gdf.loc[mask, col] = ... rather than gdf[col][mask] = .... Chained indexing can write to a temporary copy and silently do nothing (and raises SettingWithCopyWarning). On pandas 3.x with Copy-on-Write, chained assignment never works, so the .loc habit is mandatory.
Preserve nullable dtypes deliberately
After coercion you may want Int64 (nullable integer) instead of float64 so counts stay integers despite NaNs: gdf["rating"] = gdf["rating"].astype("Int64"). The nullable string dtype likewise keeps <NA> distinct from the literal string "nan", which matters when you unify null tokens.
Internal links
- What Is a GeoDataFrame? Structure and Key Concepts
- How to Read a CSV with Coordinates as a GeoDataFrame
- How to Handle Missing and Null Values in Spatial Datasets in Python
- The Python GIS Data Cleaning Checklist
- How to Read and Write GeoPackage Files in Python
FAQ
Does cleaning attribute columns ever change the geometry?
No, as long as you never include the geometry column in your transforms. String and numeric operations apply only to the columns you name or select, and select_dtypes(include=["object", "string"]) does not return the geometry dtype. Always exclude gdf.geometry.name explicitly as a safety net, then verify CRS and geometry types afterward.
Why is my number column an object dtype?
Because at least one value cannot be stored as a number β commonly an empty string, a token like "N/A", or a thousands separator such as "1,234". pandas falls back to text for the whole column β object in pandas 2, str in pandas 3. Run pd.to_numeric(col, errors="coerce") after stripping separators to convert it and turn the unparseable cells into NaN.
How do I convert messy text into a real NaN?
Build a set of the missing-value tokens your data uses ("NA", "N/A", "", "null", "-", etc.), then use s.mask(s.str.lower().isin(tokens)) on the nullable string dtype. That replaces each matching value with <NA>, after which isna(), dropna(), and fillna() all work as expected.
What is the difference between object and string dtype?
object is pandas' generic container for any Python object, and was what text read from a file became until pandas 3. Pandas 3 reads text as str, a dedicated string dtype that still marks missing values with NaN; .astype("string") gives the nullable variant with a proper <NA>, which is what the cleaning code above uses before masking null tokens.
How do I rename columns without breaking the geometry?
Renaming is safe: GeoPandas tracks the active geometry by its column name, so reassigning gdf.columns only relabels it. If your cleanup renames the geometry column itself, re-establish it with gdf = gdf.set_geometry("new_name") so gdf.geometry keeps pointing at the right column.
Should I use Int64 or float64 for integer columns with missing values?
Use Int64 (capital I, the nullable integer dtype) when you need values to stay integers despite missing entries β for example ID codes or counts. Plain float64 will display 1.0 and admits floating-point fuzz. Coerce first with pd.to_numeric(..., errors="coerce"), then .astype("Int64").
How do I fix garbled characters like ΓΒ©?
That is an encoding mismatch, so fix it when reading rather than cleaning afterward. Pass the correct encoding to gpd.read_file(path, encoding="utf-8"), or latin-1 for older shapefiles. Check the shapefile's .cpg sidecar β if it declares the wrong encoding, override it explicitly on read.
Why does my assignment seem to do nothing?
You are probably hitting chained assignment, like gdf[col][mask] = value, which writes to a temporary copy. Use gdf.loc[mask, col] = value instead, and start from a .copy() if your GeoDataFrame is a slice of another. On pandas 3.x with Copy-on-Write, the .loc form is the only one that works.