How to Fuzzy-Match Place Names When Joining Spatial Data in Python

Problem statement

You have boundaries from one source and statistics from another, and the join key is a name. The merge returns half the rows you expected:

>>> merged = boundaries.merge(stats, on="name", how="left")
>>> merged["value"].isna().sum()
143

The unmatched names are not wrong, just written differently:

boundaries              stats
────────────────────    ────────────────────
St. Albans              St Albans
Kingston upon Hull      Kingston-upon-Hull
Newcastle upon Tyne     Newcastle Upon Tyne
Bristol, City of        City of Bristol
Rhondda Cynon Taf       Rhondda Cynon Taff
München                 Muenchen

An exact string join demands byte equality. Real place names carry punctuation, abbreviations, word order, accents, case, and honest disagreements about spelling. Fuzzy matching bridges the gap — carefully, because a wrong match is worse than no match: it silently attributes one place's numbers to another.

Typical causes of a failed name join:

punctuation and abbreviations (St. vs St, hyphens, apostrophes)
differing case and leading or trailing whitespace
word order (Bristol, City of vs City of Bristol)
accents and transliteration (München, Muenchen, Munchen)
suffixes that one source includes (County, District, LGA)
Unicode normalisation differences that are invisible on screen

Quick answer

Normalise first, then fuzzy-match only what is left, then verify spatially:

normalise both sides: casefold, strip accents, drop punctuation, collapse whitespace
join on the normalised key — this alone usually fixes most of the misses
fuzzy-match the remainder with rapidfuzz, keeping the score
accept above a high threshold, review the middle band, reject the rest
confirm the accepted matches spatially where you can

import re, unicodedata
import pandas as pd

def norm_name(s: str) -> str:
    if not isinstance(s, str):
        return ""
    s = unicodedata.normalize("NFKD", s)
    s = "".join(c for c in s if not unicodedata.combining(c))     # strip accents
    s = s.lower()
    s = re.sub(r"\bst\.?\b", "saint", s)                          # common abbreviations
    s = re.sub(r"[^\w\s]", " ", s)                                # punctuation → space
    s = re.sub(r"\s+", " ", s).strip()
    return s

boundaries["key"] = boundaries["name"].map(norm_name)
stats["key"] = stats["name"].map(norm_name)

merged = boundaries.merge(stats, on="key", how="left", suffixes=("", "_stats"))
print("still unmatched:", merged["value"].isna().sum())

Normalisation is deterministic and reviewable; fuzzy matching is neither. Do as much as possible with the first, and use the second only on the leftovers.

Where a name join loses rows

Triage table of name mismatch causes and their normalisation fixes. — Six ways two spellings of the same place differ — five are fixable without fuzzy logic.

Step-by-step solution

Vertical steps: normalise, exact join, fuzzy candidates, threshold review, spatial verification. — Fuzzy matching is one stage of five — and it should see the fewest rows possible.

Measure the gap before touching anything

left = set(boundaries["name"])
right = set(stats["name"])

print(f"boundaries: {len(left)}   stats: {len(right)}")
print(f"exact matches: {len(left & right)}")
print("unmatched boundaries:", sorted(left - right)[:10])
print("unmatched stats     :", sorted(right - left)[:10])

Reading ten unmatched names tells you which normalisation rules you actually need. Do not guess: a dataset of German municipalities needs different rules from one of US counties.

Normalise both sides identically

The normalisation function must be one function used on both frames — two similar functions drift apart.

import re, unicodedata

ABBREV = {
    r"\bst\.?\b": "saint",
    r"\bmt\.?\b": "mount",
    r"\bft\.?\b": "fort",
    r"\b&\b": "and",
}
NOISE = {"county", "district", "borough", "city of", "the"}

def norm_name(s: str, drop_noise: bool = False) -> str:
    if not isinstance(s, str):
        return ""
    s = unicodedata.normalize("NFKD", s)
    s = "".join(c for c in s if not unicodedata.combining(c))
    s = s.lower().replace("ß", "ss")
    for pattern, repl in ABBREV.items():
        s = re.sub(pattern, repl, s)
    s = re.sub(r"[^\w\s]", " ", s)
    s = re.sub(r"\s+", " ", s).strip()
    if drop_noise:
        s = " ".join(w for w in s.split() if w not in NOISE)
    return s

Handle reordering with a sorted-token key, which makes bristol city of and city of bristol identical:

def token_key(s: str) -> str:
    return " ".join(sorted(norm_name(s, drop_noise=True).split()))

Join on the normalised key first

boundaries["key"] = boundaries["name"].map(norm_name)
stats["key"] = stats["name"].map(norm_name)

# check for duplicates before merging — they multiply rows silently
for df, label in ((boundaries, "boundaries"), (stats, "stats")):
    dupes = df["key"].duplicated().sum()
    if dupes:
        print(f"! {label}: {dupes} duplicate keys, e.g. "
              f"{df.loc[df['key'].duplicated(keep=False), 'name'].head(4).tolist()}")

merged = boundaries.merge(stats, on="key", how="left", suffixes=("", "_stats"))
unmatched = merged[merged["value"].isna()]
print(f"{len(unmatched)} of {len(merged)} still unmatched")

Duplicate keys are the reason a "fixed" join suddenly returns more rows than the left frame. Check every time.

Fuzzy-match only the leftovers

rapidfuzz is fast, MIT-licensed and has a clean scorer set.

from rapidfuzz import process, fuzz

candidates = stats["key"].tolist()

def best_match(key: str):
    hit = process.extractOne(key, candidates, scorer=fuzz.token_sort_ratio)
    return pd.Series(hit[:2] if hit else [None, 0], index=["match_key", "score"])

scored = unmatched["key"].apply(best_match)
review = pd.concat(
    [unmatched[["name"]].reset_index(drop=True), scored.reset_index(drop=True)], axis=1
)
print(review.sort_values("score", ascending=False).head(15).to_string(index=False))

Scorer choice matters:

fuzz.ratio — plain edit distance; good for typos
fuzz.token_sort_ratio — order-insensitive; good for City of X vs X, City of
fuzz.token_set_ratio — ignores extra words; powerful and the easiest to over-trust
fuzz.WRatio — a weighted combination; a reasonable default

Choose thresholds deliberately, and keep a review band

AUTO, REVIEW = 95, 85

review["decision"] = pd.cut(
    review["score"], bins=[-1, REVIEW, AUTO, 101],
    labels=["reject", "review", "auto"],
)
print(review["decision"].value_counts())

review.query("decision == 'review'").to_csv("data/out/name_review.csv", index=False)

The middle band is the point of the exercise. A single threshold either accepts wrong matches or rejects good ones; a review file lets a human resolve the ambiguous 20 in five minutes, and the decisions can be stored as a lookup for next time.

Verify accepted matches spatially

This is the check that a pure string workflow cannot do — and the reason to fuzzy-match spatial data rather than plain tables.

import geopandas as gpd

# stats carry a representative point, or you have a second boundary source
pts = gpd.GeoDataFrame(
    stats, geometry=gpd.points_from_xy(stats["lon"], stats["lat"]), crs="EPSG:4326"
).to_crs(boundaries.crs)

check = gpd.sjoin(pts, boundaries[["name", "geometry"]], how="left", predicate="within")
disagree = check[check["name_left"].map(norm_name) != check["name_right"].map(norm_name)]
print(f"{len(disagree)} matches where the point falls in a differently named polygon")

A name match whose point lands in the wrong polygon is almost certainly wrong, however high the score.

Store the resolved pairs as a lookup

import json
from pathlib import Path

accepted = review.query("decision == 'auto'")[["name", "match_key"]]
lookup = dict(zip(accepted["name"], accepted["match_key"]))
Path("data/ref/name_lookup.json").write_text(json.dumps(lookup, indent=2, ensure_ascii=False),
                                              encoding="utf-8")

Next month's run applies the lookup first and only fuzzy-matches genuinely new names. Matching becomes cheaper and more stable every time it runs.

Code examples

Example 1: a complete, auditable name join

import json, re, unicodedata
from pathlib import Path
import geopandas as gpd
import pandas as pd
from rapidfuzz import process, fuzz

AUTO, REVIEW = 95, 85
LOOKUP = Path("data/ref/name_lookup.json")

def norm_name(s: str) -> str:
    if not isinstance(s, str):
        return ""
    s = unicodedata.normalize("NFKD", s)
    s = "".join(c for c in s if not unicodedata.combining(c)).lower()
    s = re.sub(r"\bst\.?\b", "saint", s)
    s = re.sub(r"[^\w\s]", " ", s)
    return re.sub(r"\s+", " ", s).strip()

def join_by_name(left: gpd.GeoDataFrame, right: pd.DataFrame,
                 left_col="name", right_col="name"):
    left = left.copy(); right = right.copy()
    left["key"] = left[left_col].map(norm_name)
    right["key"] = right[right_col].map(norm_name)

    manual = json.loads(LOOKUP.read_text(encoding="utf-8")) if LOOKUP.exists() else {}
    left["key"] = left.apply(lambda r: norm_name(manual.get(r[left_col], r[left_col])), axis=1)

    out = left.merge(right.drop(columns=[right_col]), on="key", how="left", indicator=True)
    missing = out["_merge"] == "left_only"

    suggestions = []
    if missing.any():
        pool = right["key"].tolist()
        for name, key in zip(out.loc[missing, left_col], out.loc[missing, "key"]):
            hit = process.extractOne(key, pool, scorer=fuzz.token_sort_ratio)
            if hit:
                suggestions.append({"name": name, "suggested": hit[0], "score": round(hit[1], 1)})

    report = {
        "rows": len(out),
        "matched": int((~missing).sum()),
        "unmatched": int(missing.sum()),
        "auto_candidates": sum(1 for s in suggestions if s["score"] >= AUTO),
        "review_candidates": sum(1 for s in suggestions if REVIEW <= s["score"] < AUTO),
    }
    return out.drop(columns="_merge"), pd.DataFrame(suggestions), report

joined, suggestions, report = join_by_name(boundaries, stats)
print(report)
suggestions.sort_values("score", ascending=False).to_csv("data/out/name_review.csv", index=False)

Example 2: block the comparison so it scales

Comparing 5,000 names against 5,000 is 25 million string comparisons. Blocking on a coarse key cuts that by orders of magnitude.

from rapidfuzz import process, fuzz

right_by_region = {r: g["key"].tolist() for r, g in stats.groupby("region")}

def match_within_region(row):
    pool = right_by_region.get(row["region"], [])
    if not pool:
        return None, 0
    hit = process.extractOne(row["key"], pool, scorer=fuzz.token_sort_ratio)
    return (hit[0], hit[1]) if hit else (None, 0)

unmatched[["match_key", "score"]] = unmatched.apply(
    match_within_region, axis=1, result_type="expand"
)

Blocking on a region, a first letter, or a postcode district also removes a whole class of wrong matches: two similarly named places on opposite sides of the country can no longer be confused.

Example 3: match by geometry instead, and use names to confirm

When both sides are spatial, the geometry is the better key.

import geopandas as gpd

a = gpd.read_file("data/raw/boundaries_a.gpkg").to_crs(3857)
b = gpd.read_file("data/raw/boundaries_b.gpkg").to_crs(3857)

a["rep"] = a.representative_point()
pairs = gpd.sjoin(a.set_geometry("rep"), b[["name", "geometry"]],
                  how="left", predicate="within")

pairs["name_agrees"] = (pairs["name_left"].map(norm_name) == pairs["name_right"].map(norm_name))
print(pairs["name_agrees"].value_counts())
print(pairs.loc[~pairs["name_agrees"], ["name_left", "name_right"]].head(10).to_string(index=False))

Where geometry and name disagree, you have found either a genuine boundary change or a bad record — both worth knowing about.

Example 4: a one-to-one assignment, not a greedy best match

Greedy matching can map two different left names onto the same right name. When the relationship should be one-to-one, solve it as an assignment problem.

import numpy as np
from rapidfuzz import process, fuzz
from scipy.optimize import linear_sum_assignment

left_keys = unmatched["key"].tolist()
right_keys = stats["key"].tolist()

scores = process.cdist(left_keys, right_keys, scorer=fuzz.token_sort_ratio)
row_idx, col_idx = linear_sum_assignment(-np.asarray(scores))

pairs = [(left_keys[r], right_keys[c], scores[r][c]) for r, c in zip(row_idx, col_idx)]
for l, r, s in sorted(pairs, key=lambda p: -p[2])[:10]:
    print(f"{s:5.1f}  {l}  →  {r}")

Explanation

An exact join is a hash lookup: two keys match only if their bytes are identical. Place names fail that test constantly, because a name is written by people for people — with punctuation choices, abbreviations, local variants and typing errors — and the two datasets you are joining were written by different people at different times.

Grid of score bands from auto-accept through review to reject, with what belongs in each. — Three bands, not one threshold — the middle band is where the human belongs.

Normalisation removes the differences that carry no meaning. Casefolding, accent stripping, punctuation removal and whitespace collapsing are all lossless in the sense that matters: two strings that differ only in those respects genuinely refer to the same place. Because the transformation is deterministic, you can inspect it, test it, and re-run it and get the same answer — which is why it should do as much of the work as possible.

Fuzzy matching is different in kind. It computes a similarity score, and a score is not a decision. Newport and Newry score high; so do East Riding and West Riding, which are different places. The score also has no notion of population, geography or context, so on its own it will confidently produce wrong matches at exactly the rate its threshold implies. Treating the score as three bands — accept, review, reject — makes that uncertainty explicit instead of hiding it inside a single number.

The advantage you have with spatial data is a second, independent signal. If both datasets have geometry, a representative point falling inside the other's polygon is far stronger evidence than any string score, and can be used to confirm or overturn a fuzzy match. Even a single coordinate pair per statistical record is enough. Where names and geometry agree, the match is safe; where they disagree, you have found something worth a human's attention — which is exactly what a review file is for.

Finally, persist your decisions. Manual resolutions stored as a lookup turn a recurring problem into a shrinking one: the next run starts with the answers you already worked out, and only genuinely new names reach the fuzzy stage.

Edge cases or notes

token_set_ratio is over-permissive: It ignores extra words entirely, so Newcastle scores 100 against Newcastle upon Tyne. Use it knowingly, not as a default.
Duplicate keys multiply rows: Check key.duplicated() on both frames before merging, or a "fix" will inflate the row count.
Normalisation must be identical on both sides: One shared function, applied twice. Two similar functions drift.
Accent stripping is not always right: In some datasets Åland and Aland are genuinely different entries. Test on your data before adopting a rule.
Prefer stable codes when they exist: ONS/GSS, FIPS, NUTS, ISO codes and OSM ids beat any name match. Fuzzy matching is the fallback, not the plan.
Names change over time: Boundary reorganisations rename and merge areas. A no-match may be a real historical change, not a spelling difference.
rapidfuzz vs fuzzywuzzy: rapidfuzz is much faster, actively maintained, and MIT-licensed. There is no reason to reach for the older library.

Internal links

FAQ

Which library should I use for fuzzy matching in Python?

rapidfuzz. It is a fast, MIT-licensed implementation with the same scorer names as the older fuzzywuzzy, plus process.cdist for bulk comparisons. difflib in the standard library works for small jobs.

What threshold should I use?

There is no universal number — it depends on name length and how similar your candidates are. Start with auto-accept at 95, review between 85 and 95, reject below, then tune after looking at the review file once.

Why does my fuzzy join produce more rows than I started with?

Duplicate keys on one side, or several left names matching the same right name. Check duplicated() on both keys before merging, and use an assignment solver when the relationship must be one-to-one.

Is normalisation enough on its own?

Very often, yes. Casefolding, accent stripping, punctuation removal and token sorting typically resolve the large majority of misses, leaving only a handful for fuzzy matching — which is exactly where you want to be.

How do I verify a fuzzy match is correct?

Use geometry. If a representative point of one record falls inside the other's polygon, the match is well supported. Where the two signals disagree, send the pair to manual review.

Can I match on more than the name?

Yes, and you should when you can. Blocking by region, county or postcode before matching removes wrong candidates entirely and makes the comparison much faster.

How do I stop redoing this work every month?

Persist accepted pairs as a JSON or CSV lookup and apply it before matching. Each run then only fuzzy-matches genuinely new names, and the manual effort trends towards zero.

How to Fuzzy-Match Place Names When Joining Spatial Data in Python #

Problem statement #

Quick answer #

Where a name join loses rows #

Step-by-step solution #

Measure the gap before touching anything #

Normalise both sides identically #

Join on the normalised key first #

Fuzzy-match only the leftovers #

Choose thresholds deliberately, and keep a review band #

Verify accepted matches spatially #

Store the resolved pairs as a lookup #

Code examples #

Example 1: a complete, auditable name join #

Example 2: block the comparison so it scales #

Example 3: match by geometry instead, and use names to confirm #

Example 4: a one-to-one assignment, not a greedy best match #

Explanation #

Edge cases or notes #

Internal links #

FAQ #

Which library should I use for fuzzy matching in Python? #

What threshold should I use? #

Why does my fuzzy join produce more rows than I started with? #

Is normalisation enough on its own? #

How do I verify a fuzzy match is correct? #

Can I match on more than the name? #

How do I stop redoing this work every month? #