How to Fuzzy-Match Place Names When Joining Spatial Data in Python

On this page

Problem statement

You have boundaries from one source and statistics from another, and the join key is a name. The merge returns half the rows you expected:

merged = boundaries.merge(stats, on="name", how="left")
print(merged["value"].isna().sum(), "of", len(merged), "districts unmatched")
print(sorted(set(boundaries["name"]) - set(stats["name"]))[:8])
print(sorted(set(stats["name"]) - set(boundaries["name"]))[:8])
36 of 361 districts unmatched
['Belfast', 'Blackburn with Darwen', 'Bristol, City of', 'Cambridge', 'Canterbury', 'City of Edinburgh', 'City of London', 'County Durham']
['Anglesey', 'Belfast City', 'Blackburn', 'Bristol', 'Cambridge City', 'Canterbury City', 'City of London Corporation', 'Comhairle nan Eilean Siar (Western Isles)']

Here the boundaries are the UK's 361 local authority districts from the ONS and the statistics are the Food Standards Agency's list of local authorities with the number of food businesses each inspects. The unmatched names are not wrong, just written differently — "Bristol, City of" against "Bristol", "St. Helens" against "St Helens", "Newcastle upon Tyne" against "Newcastle Upon Tyne", "Cambridge" against "Cambridge City" — and a few are not the same kind of thing at all: the FSA list includes port health authorities such as "Hull and Goole Port" that have no district.

An exact string join demands byte equality. Real place names carry punctuation, abbreviations, word order, accents, case, and honest disagreements about spelling. Fuzzy matching bridges the gap — carefully, because a wrong match is worse than no match: it silently attributes one place's numbers to another.

Typical causes of a failed name join:

  • punctuation and abbreviations (St. vs St, hyphens, apostrophes)
  • differing case and leading or trailing whitespace
  • word order (Bristol, City of vs City of Bristol)
  • accents and transliteration (München, Muenchen, Munchen)
  • suffixes that one source includes (County, District, LGA)
  • Unicode normalisation differences that are invisible on screen

Quick answer

Normalise first, then fuzzy-match only what is left, then verify spatially:

  1. normalise both sides: casefold, strip accents, drop punctuation, collapse whitespace
  2. join on the normalised key — here it fixed 8 of the 36 misses, and dropping filler words such as "City" and "Council" fixed another 24
  3. fuzzy-match the remainder with rapidfuzz, keeping the score
  4. accept above a high threshold, review the middle band, reject the rest
  5. confirm the accepted matches spatially where you can
import re, unicodedata
import pandas as pd

def norm_name(s: str) -> str:
    if not isinstance(s, str):
        return ""
    s = unicodedata.normalize("NFKD", s)
    s = "".join(c for c in s if not unicodedata.combining(c))     # strip accents
    s = s.lower()
    s = re.sub(r"\bst\.?\b", "saint", s)                          # common abbreviations
    s = re.sub(r"[^\w\s]", " ", s)                                # punctuation → space
    s = re.sub(r"\s+", " ", s).strip()
    return s

boundaries["key"] = boundaries["name"].map(norm_name)
stats["key"] = stats["name"].map(norm_name)

merged = boundaries.merge(stats, on="key", how="left", suffixes=("", "_stats"))
print("still unmatched:", merged["value"].isna().sum())
still unmatched: 28

Normalisation is deterministic and reviewable; fuzzy matching is neither. Do as much as possible with the first, and use the second only on the leftovers. On this pair of sources, the sorted-token key below — normalised, filler words dropped — matched 357 of the 361 districts before any fuzzy matching at all.

Where a name join loses rows

Six reasons a name join fails, with the normalisation that fixes each. six ways two spellings differWhy it did not matchFixSt. Albans vs St AlbansExpand abbreviations, droppunctuationKingston-upon-Hull vs spacesPunctuation to spaces, collapsethemBristol, City of vs City of…Sort the tokens into a stable keyMünchen vs MuenchenStrip accents, normalise to NFCTrailing spaces and casestrip() and casefold() both sidesCounty / District suffixesDrop known noise wordsFive of the six need no fuzzy logic at all spatialworkflow.io
Six ways two spellings of the same place differ — five are fixable without fuzzy logic.

Normalise, join exactly, then fuzzy-match the leftovers

The matching pipeline from normalisation to spatial verification. fuzzy matching sees the fewest rows1normalise both sidesdeterministic2join on the clean keymost matches land3fuzzy-score the remainderrapidfuzz4accept / review / rejectthree bands5verify spatiallypoint in polygon6save the resolved pairsa shrinking problemPersisting decisions means next month starts with the answers spatialworkflow.io
Fuzzy matching is one stage of five — and it should see the fewest rows possible.

Measure the gap before touching anything

left = set(boundaries["name"])
right = set(stats["name"])

print(f"boundaries: {len(left)}   stats: {len(right)}")
print(f"exact matches: {len(left & right)}")
print("unmatched boundaries:", sorted(left - right)[:10])
print("unmatched stats     :", sorted(right - left)[:10])
boundaries: 361   stats: 363
exact matches: 325
unmatched boundaries: ['Belfast', 'Blackburn with Darwen', 'Bristol, City of', 'Cambridge', 'Canterbury', 'City of Edinburgh', 'City of London', 'County Durham', 'Derby', 'Exeter']
unmatched stats     : ['Anglesey', 'Belfast City', 'Blackburn', 'Bristol', 'Cambridge City', 'Canterbury City', 'City of London Corporation', 'Comhairle nan Eilean Siar (Western Isles)', 'Derby City', 'Durham']

Reading ten unmatched names tells you which normalisation rules you actually need. Do not guess: these two lists say that one source appends "City" and "Council", and the other writes "Bristol, City of" — rules no German or US dataset would need.

Normalise both sides identically

The normalisation function must be one function used on both frames — two similar functions drift apart.

import re, unicodedata

ABBREV = {
    r"\bst\.?\b": "saint",
    r"\bmt\.?\b": "mount",
    r"\bft\.?\b": "fort",
    r"\b&\b": "and",
}
NOISE = {"county", "district", "borough", "city", "council", "corporation", "of", "the"}

def norm_name(s: str, drop_noise: bool = False) -> str:
    if not isinstance(s, str):
        return ""
    s = unicodedata.normalize("NFKD", s)
    s = "".join(c for c in s if not unicodedata.combining(c))
    s = s.lower().replace("ß", "ss")
    for pattern, repl in ABBREV.items():
        s = re.sub(pattern, repl, s)
    s = re.sub(r"[^\w\s]", " ", s)
    s = re.sub(r"\s+", " ", s).strip()
    if drop_noise:
        s = " ".join(w for w in s.split() if w not in NOISE)
    return s

NOISE is compared word by word, so it must hold single words. An earlier version of this page listed "city of", which can never equal one word, and so removed nothing.

Handle reordering with a sorted-token key, which makes bristol city of and city of bristol identical:

def token_key(s: str) -> str:
    return " ".join(sorted(norm_name(s, drop_noise=True).split()))

for label, key in (("norm_name", norm_name), ("token_key", token_key)):
    matched = boundaries["name"].map(key).isin(set(stats["name"].map(key))).sum()
    print(f"{label:9} matches {matched} of {len(boundaries)}")
norm_name matches 333 of 361
token_key matches 357 of 361

Join on the normalised key first

boundaries["key"] = boundaries["name"].map(norm_name)
stats["key"] = stats["name"].map(norm_name)

# check for duplicates before merging — they multiply rows silently
for df, label in ((boundaries, "boundaries"), (stats, "stats")):
    dupes = df["key"].duplicated().sum()
    if dupes:
        print(f"! {label}: {dupes} duplicate keys, e.g. "
              f"{df.loc[df['key'].duplicated(keep=False), 'name'].head(4).tolist()}")

merged = boundaries.merge(stats, on="key", how="left", suffixes=("", "_stats"))
unmatched = merged[merged["value"].isna()]
print(f"{len(unmatched)} of {len(merged)} still unmatched")
28 of 361 still unmatched

Duplicate keys are the reason a "fixed" join suddenly returns more rows than the left frame. Check every time.

ONS local authority districts left unmatched to the FSA authority list after each normalisation step. 361 districts, unmatched after each stepexact names36 unmatchednormalisenorm_name28drop City, Council…÷ 7token_key4Left for review: Blackburn with Darwen, Kingston upon Hull, Na h-Eileanan Siar, Isle of Anglesey spatialworkflow.io
Deterministic rules resolved 32 of 36 misses; fuzzy matching should only ever see what is left.

Fuzzy-match only the leftovers

rapidfuzz is fast, MIT-licensed and has a clean scorer set.

from rapidfuzz import process, fuzz

candidates = stats["key"].tolist()

def best_match(key: str):
    hit = process.extractOne(key, candidates, scorer=fuzz.token_sort_ratio)
    return pd.Series(hit[:2] if hit else [None, 0], index=["match_key", "score"])

scored = unmatched["key"].apply(best_match)
review = pd.concat(
    [unmatched[["name"]].reset_index(drop=True), scored.reset_index(drop=True)], axis=1
)
print(review.sort_values("score", ascending=False).head(15).to_string(index=False))
                   name                    match_key      score
      City of Edinburgh            edinburgh city of 100.000000
Lisburn and Castlereagh lisburn and castlereagh city  90.196078
           Peterborough            peterborough city  82.758621
              St Albans            saint albans city  82.758621
     Telford and Wrekin   telford and wrekin council  81.818182
             Gloucester              gloucester city  80.000000
             Winchester                   chichester  80.000000
             Canterbury              canterbury city  80.000000
             Nottingham              nottingham city  80.000000
              Worcester               worcester city  78.260870
              Cambridge               cambridge city  78.260870
              Leicester               leicester city  78.260870
              Lancaster               lancaster city  78.260870
               Plymouth                plymouth city  76.190476
                Belfast                 belfast city  73.684211

Scorer choice matters:

  • fuzz.ratio — plain edit distance; good for typos
  • fuzz.token_sort_ratio — order-insensitive; good for City of X vs X, City of
  • fuzz.token_set_ratio — ignores extra words; powerful and the easiest to over-trust
  • fuzz.WRatio — a weighted combination; a reasonable default

Choose thresholds deliberately, and keep a review band

AUTO, REVIEW = 95, 85

review["decision"] = pd.cut(
    review["score"], bins=[-1, REVIEW, AUTO, 101],
    labels=["reject", "review", "auto"],
)
print(review["decision"].value_counts())

review.query("decision == 'review'").to_csv("data/out/name_review.csv", index=False)
decision
reject    26
review     1
auto       1
Name: count, dtype: int64

The middle band is the point of the exercise. A single threshold either accepts wrong matches or rejects good ones; a review file lets a human resolve the ambiguous ones in minutes, and the decisions can be stored as a lookup for next time.

Look at what the scores did here, though. With the FSA's "City" suffix still attached, correct pairs such as gloucester → gloucester city score 80 — exactly what the wrong pair winchester → chichester scores — so 26 of the 28 leftovers fall below the review band, right and wrong together. No threshold separates them. The fix was upstream, in the normalisation: once "city", "council" and "corporation" are dropped as filler, those pairs match exactly and never reach the fuzzy stage.

Verify accepted matches spatially

This is the check that a pure string workflow cannot do — and the reason to fuzzy-match spatial data rather than plain tables.

import geopandas as gpd

# stats carry a representative point, or you have a second boundary source
pts = gpd.GeoDataFrame(
    stats, geometry=gpd.points_from_xy(stats["lon"], stats["lat"]), crs="EPSG:4326"
).to_crs(boundaries.crs)

check = gpd.sjoin(pts, boundaries[["name", "geometry"]], how="left", predicate="within")
disagree = check[check["name_left"].map(token_key) != check["name_right"].map(token_key)]
print(f"{len(disagree)} of {len(check)} FSA authorities whose point falls in a differently named district")
print(disagree[["name_left", "name_right"]].head(8).to_string(index=False))
13 of 363 FSA authorities whose point falls in a differently named district
                                name_left                  name_right
                                 Anglesey            Isle of Anglesey
                                Blackburn       Blackburn with Darwen
                                 Bolsover                    Ashfield
Comhairle nan Eilean Siar (Western Isles)                         NaN
                      East Dunbartonshire                Glasgow City
                        East Renfrewshire                Glasgow City
                      Hull and Goole Port Kingston upon Hull, City of
                                Hull City Kingston upon Hull, City of

A name match whose point lands in a differently named polygon deserves a look, however high the score. The 13 here are a mixture: naming differences a matcher should resolve (Anglesey against "Isle of Anglesey"), FSA authorities that are not districts at all ("Hull and Goole Port", a port health authority), a representative point that fell in the sea (the Western Isles), and points that are simply poor representatives — the median of 25 East Dunbartonshire businesses lands across the boundary in Glasgow. A disagreement is a reason to review, not a verdict.

Store the resolved pairs as a lookup

import json
from pathlib import Path

accepted = review.query("decision == 'auto'")[["name", "match_key"]]
lookup = dict(zip(accepted["name"], accepted["match_key"]))
Path("data/ref/name_lookup.json").write_text(json.dumps(lookup, indent=2, ensure_ascii=False),
                                              encoding="utf-8")
print(len(lookup), "pairs stored")
1 pairs stored

Next month's run applies the lookup first and only fuzzy-matches genuinely new names. Matching becomes cheaper and more stable every time it runs.

Joining FSA authorities to ONS district names

The data used on this page

  • boundaries: the UK's 361 local authority districts, December 2024, from the ONS Open Geography Portal, with the ONS name and code, and a region — the nation, from the code's first letter.
  • stats: the Food Standards Agency's 363 local authorities, with value the number of food businesses each registers, the FSA's region reduced to the same nations, and lon/lat the median location of 25 of its businesses, fetched from the FSA's API.
  • boundaries_a.gpkg and boundaries_b.gpkg: the districts of December 2024 and December 2021, for matching by geometry across a reorganisation.
import os
import statistics
import time

import geopandas as gpd
import pandas as pd
import requests

os.makedirs("data/raw", exist_ok=True)
os.makedirs("data/out", exist_ok=True)
os.makedirs("data/ref", exist_ok=True)
ONS = "https://services1.arcgis.com/ESMARspQHYMw9BZ9/arcgis/rest/services/{}/FeatureServer/0/query"
NATION = {"E": "England", "W": "Wales", "S": "Scotland", "N": "Northern Ireland"}


def ons_districts(service, code_field, name_field):
    features, offset = [], 0
    while True:
        page = requests.get(ONS.format(service), timeout=300, params={
            "where": "1=1", "outFields": f"{code_field},{name_field}", "outSR": 4326, "f": "geojson",
            "resultOffset": offset, "resultRecordCount": 50}).json()["features"]
        features += page
        offset += 50
        if len(page) < 50:
            return gpd.GeoDataFrame.from_features(features, crs=4326).rename(
                columns={code_field: "code", name_field: "name"})


if not os.path.exists("data/raw/boundaries_a.gpkg"):
    ons_districts("Local_Authority_Districts_December_2024_Boundaries_UK_BGC", "LAD24CD", "LAD24NM").to_file(
        "data/raw/boundaries_a.gpkg")
    ons_districts("Local_Authority_Districts_December_2021_UK_BGC_2022", "LAD21CD", "LAD21NM").to_file(
        "data/raw/boundaries_b.gpkg")
if not os.path.exists("data/raw/stats.csv"):
    fsa = requests.get("https://api.ratings.food.gov.uk/Authorities", headers={"x-api-version": "2"},
                       timeout=120).json()["authorities"]
    session, rows = requests.Session(), []

    def establishments(authority_id, tries=5):
        for attempt in range(tries):                 # the API throttles bursts: back off and retry
            reply = session.get("https://api.ratings.food.gov.uk/Establishments", timeout=60,
                                params={"localAuthorityId": authority_id, "pageSize": 25},
                                headers={"x-api-version": "2"})
            if reply.ok and reply.text.startswith("{"):
                return reply.json()["establishments"]
            time.sleep(2 ** attempt)
        return []

    for a in fsa:
        page = establishments(a["LocalAuthorityId"])
        xy = [(float(e["geocode"]["longitude"]), float(e["geocode"]["latitude"])) for e in page
              if e.get("geocode") and e["geocode"].get("latitude")]
        region = a["RegionName"] if a["RegionName"] in NATION.values() else "England"
        rows.append({"name": a["Name"], "region": region, "value": a["EstablishmentCount"],
                     "lon": statistics.median(x for x, _ in xy) if xy else None,
                     "lat": statistics.median(y for _, y in xy) if xy else None})
    pd.DataFrame(rows).to_csv("data/raw/stats.csv", index=False)

boundaries = gpd.read_file("data/raw/boundaries_a.gpkg")
boundaries["region"] = boundaries["code"].str[0].map(NATION)
stats = pd.read_csv("data/raw/stats.csv").dropna(subset=["lon", "lat"])
print(len(boundaries), "districts;", len(stats), "FSA authorities with a location;",
      stats["region"].value_counts().to_dict())
361 districts; 363 FSA authorities with a location; {'England': 298, 'Scotland': 32, 'Wales': 22, 'Northern Ireland': 11}

Example 1: a complete, auditable name join

import json, re, unicodedata
from pathlib import Path
import geopandas as gpd
import pandas as pd
from rapidfuzz import process, fuzz

AUTO, REVIEW = 95, 85
LOOKUP = Path("data/ref/name_lookup.json")

def norm_name(s: str) -> str:
    if not isinstance(s, str):
        return ""
    s = unicodedata.normalize("NFKD", s)
    s = "".join(c for c in s if not unicodedata.combining(c)).lower()
    s = re.sub(r"\bst\.?\b", "saint", s)
    s = re.sub(r"[^\w\s]", " ", s)
    return re.sub(r"\s+", " ", s).strip()

def join_by_name(left: gpd.GeoDataFrame, right: pd.DataFrame,
                 left_col="name", right_col="name"):
    left = left.copy(); right = right.copy()
    left["key"] = left[left_col].map(norm_name)
    right["key"] = right[right_col].map(norm_name)

    manual = json.loads(LOOKUP.read_text(encoding="utf-8")) if LOOKUP.exists() else {}
    left["key"] = left.apply(lambda r: norm_name(manual.get(r[left_col], r[left_col])), axis=1)

    out = left.merge(right.drop(columns=[right_col]), on="key", how="left", indicator=True)
    missing = out["_merge"] == "left_only"

    suggestions = []
    if missing.any():
        pool = right["key"].tolist()
        for name, key in zip(out.loc[missing, left_col], out.loc[missing, "key"]):
            hit = process.extractOne(key, pool, scorer=fuzz.token_sort_ratio)
            if hit:
                suggestions.append({"name": name, "suggested": hit[0], "score": round(hit[1], 1)})

    report = {
        "rows": len(out),
        "matched": int((~missing).sum()),
        "unmatched": int(missing.sum()),
        "auto_candidates": sum(1 for s in suggestions if s["score"] >= AUTO),
        "review_candidates": sum(1 for s in suggestions if REVIEW <= s["score"] < AUTO),
    }
    return out.drop(columns="_merge"), pd.DataFrame(suggestions), report

joined, suggestions, report = join_by_name(boundaries, stats)
print(report)
suggestions.sort_values("score", ascending=False).to_csv("data/out/name_review.csv", index=False)
{'rows': 361, 'matched': 334, 'unmatched': 27, 'auto_candidates': 0, 'review_candidates': 1}

Example 2: block the comparison so it scales

Comparing 5,000 names against 5,000 is 25 million string comparisons. Blocking on a coarse key cuts that by orders of magnitude.

from rapidfuzz import process, fuzz

unmatched = unmatched.copy()
right_by_region = {r: g["key"].tolist() for r, g in stats.groupby("region")}

def match_within_region(row):
    pool = right_by_region.get(row["region"], [])
    if not pool:
        return None, 0
    hit = process.extractOne(row["key"], pool, scorer=fuzz.token_sort_ratio)
    return (hit[0], hit[1]) if hit else (None, 0)

unmatched[["match_key", "score"]] = unmatched.apply(
    match_within_region, axis=1, result_type="expand"
)
print(unmatched[["name", "region", "match_key", "score"]].sort_values("score").head(6).to_string(index=False))
                       name   region            match_key     score
         Na h-Eileanan Siar Scotland     clackmannanshire 52.941176
      Blackburn with Darwen  England            blackburn 60.000000
Kingston upon Hull, City of  England kingston upon thames 60.869565
              County Durham  England               durham 63.157895
           Bristol, City of  England              bristol 63.636364
                      Derby  England           derby city 66.666667

Blocking on a region, a first letter, or a postcode district also removes a whole class of wrong matches: two similarly named places on opposite sides of the country can no longer be confused.

Example 3: match by geometry instead, and use names to confirm

When both sides are spatial, the geometry is the better key.

import geopandas as gpd

a = gpd.read_file("data/raw/boundaries_a.gpkg").to_crs(27700)     # districts, December 2024
b = gpd.read_file("data/raw/boundaries_b.gpkg").to_crs(27700)     # districts, December 2021

a["rep"] = a.representative_point()
pairs = gpd.sjoin(a.set_geometry("rep"), b[["name", "geometry"]],
                  how="left", predicate="within")

pairs["name_agrees"] = (pairs["name_left"].map(norm_name) == pairs["name_right"].map(norm_name))
print(pairs["name_agrees"].value_counts())
print(pairs.loc[~pairs["name_agrees"], ["name_left", "name_right"]].head(10).to_string(index=False))
name_agrees
True     357
False      4
Name: count, dtype: int64
              name_left                name_right
             Cumberland                 Allerdale
Westmorland and Furness                      Eden
        North Yorkshire                 Harrogate
               Somerset Somerset West and Taunton

Where geometry and name disagree, you have found either a genuine boundary change or a bad record — both worth knowing about. Here the four disagreements between the December 2021 and December 2024 districts are all the April 2023 reorganisations: Cumberland replacing Allerdale, Westmorland and Furness replacing Eden, and the new unitary North Yorkshire and Somerset.

The four places where the December 2024 and December 2021 district boundaries disagree on the name at the same location. geometry matched, names disagree: the 2023 reorganisationdistrict in 2024district in 2021 at that pointCumberlandAllerdaleWestmorland and FurnessEdenNorth YorkshireHarrogateSomersetSomerset West and Taunton357 of 361 representative points landed in a district of the same nameEach 2024 district absorbed several 2021 ones; the representative point lands in one of them spatialworkflow.io
A no-match can be history rather than spelling.

Example 4: a one-to-one assignment, not a greedy best match

Greedy matching can map two different left names onto the same right name. When the relationship should be one-to-one, solve it as an assignment problem — which enforces uniqueness, not correctness: here it still pairs winchester with chichester, because both scored 80 against everything better.

import numpy as np
from rapidfuzz import process, fuzz
from scipy.optimize import linear_sum_assignment

left_keys = unmatched["key"].tolist()
right_keys = stats["key"].tolist()

scores = process.cdist(left_keys, right_keys, scorer=fuzz.token_sort_ratio)
row_idx, col_idx = linear_sum_assignment(-np.asarray(scores))

pairs = [(left_keys[r], right_keys[c], scores[r][c]) for r, c in zip(row_idx, col_idx)]
for l, r, s in sorted(pairs, key=lambda p: -p[2])[:10]:
    print(f"{s:5.1f}  {l}  →  {r}")
100.0  city of edinburgh  →  edinburgh city of
 90.2  lisburn and castlereagh  →  lisburn and castlereagh city
 82.8  peterborough  →  peterborough city
 82.8  saint albans  →  saint albans city
 81.8  telford and wrekin  →  telford and wrekin council
 80.0  nottingham  →  nottingham city
 80.0  gloucester  →  gloucester city
 80.0  winchester  →  chichester
 80.0  canterbury  →  canterbury city
 78.3  leicester  →  leicester city

Why exact joins fail on place names

An exact join is a hash lookup: two keys match only if their bytes are identical. Place names fail that test constantly, because a name is written by people for people — with punctuation choices, abbreviations, local variants and typing errors — and the two datasets you are joining were written by different people at different times.

Score bands for fuzzy matching and what belongs in each. three bands, not one thresholddecisionwho handles itscore ≥ 95✓acceptthe script85 – 95!reviewa human, oncebelow 85✕rejectnobody — leave unmatchedA single cut-off either invents matches or throws good ones away spatialworkflow.io
Three bands, not one threshold — the middle band is where the human belongs.

Normalisation removes the differences that carry no meaning. Casefolding, accent stripping, punctuation removal and whitespace collapsing are all lossless in the sense that matters: two strings that differ only in those respects genuinely refer to the same place. Because the transformation is deterministic, you can inspect it, test it, and re-run it and get the same answer — which is why it should do as much of the work as possible.

Fuzzy matching is different in kind. It computes a similarity score, and a score is not a decision. Winchester and Chichester scored 80 on this data — the same as Gloucester against the FSA's Gloucester City. The score also has no notion of population, geography or context, so on its own it will confidently produce wrong matches at exactly the rate its threshold implies. Treating the score as three bands — accept, review, reject — makes that uncertainty explicit instead of hiding it inside a single number.

The advantage you have with spatial data is a second, independent signal. If both datasets have geometry, a representative point falling inside the other's polygon is far stronger evidence than any string score, and can be used to confirm or overturn a fuzzy match. Even a single coordinate pair per statistical record is enough. Where names and geometry agree, the match is safe; where they disagree, you have found something worth a human's attention — which is exactly what a review file is for.

Finally, persist your decisions. Manual resolutions stored as a lookup turn a recurring problem into a shrinking one: the next run starts with the answers you already worked out, and only genuinely new names reach the fuzzy stage.

Scorers and keys that over-match

  • token_set_ratio is over-permissive: It ignores extra words entirely, so Newcastle scores 100 against Newcastle upon Tyne. Use it knowingly, not as a default.
  • Duplicate keys multiply rows: Check key.duplicated() on both frames before merging, or a "fix" will inflate the row count.
  • Normalisation must be identical on both sides: One shared function, applied twice. Two similar functions drift.
  • Accent stripping is not always right: In some datasets Åland and Aland are genuinely different entries. Test on your data before adopting a rule.
  • Prefer stable codes when they exist: ONS/GSS, FIPS, NUTS, ISO codes and OSM ids beat any name match. Fuzzy matching is the fallback, not the plan.
  • Names change over time: Boundary reorganisations rename and merge areas. A no-match may be a real historical change, not a spelling difference.
  • rapidfuzz vs fuzzywuzzy: rapidfuzz is much faster, actively maintained, and MIT-licensed. There is no reason to reach for the older library.

FAQ

Which library should I use for fuzzy matching in Python?

rapidfuzz. It is a fast, MIT-licensed implementation with the same scorer names as the older fuzzywuzzy, plus process.cdist for bulk comparisons. difflib in the standard library works for small jobs.

What threshold should I use?

There is no universal number — it depends on name length and how similar your candidates are. Start with auto-accept at 95, review between 85 and 95, reject below, then tune after looking at the review file once. If correct and incorrect pairs score the same, as gloucester → gloucester city and winchester → chichester both did at 80, no threshold will separate them; improve the normalisation instead.

Why does my fuzzy join produce more rows than I started with?

Duplicate keys on one side, or several left names matching the same right name. Check duplicated() on both keys before merging, and use an assignment solver when the relationship must be one-to-one.

Is normalisation enough on its own?

Very often, yes. On the ONS districts against the FSA's authorities, casefolding, punctuation removal, dropping filler words and token sorting matched 357 of 361, leaving four for review — which is exactly where you want to be.

How do I verify a fuzzy match is correct?

Use geometry. If a representative point of one record falls inside the other's polygon, the match is well supported. Where the two signals disagree, send the pair to manual review.

Can I match on more than the name?

Yes, and you should when you can. Blocking by region, county or postcode before matching removes wrong candidates entirely and makes the comparison much faster.

How do I stop redoing this work every month?

Persist accepted pairs as a JSON or CSV lookup and apply it before matching. Each run then only fuzzy-matches genuinely new names, and the manual effort trends towards zero.