How to Parse and Normalise Addresses in Python

On this page

Problem statement

You have tens of thousands of addresses in one column and a geocoder that charges per request. Before you spend anything you need to know how many of those rows are actually distinct, which ones are missing the parts that make a match possible, and which ones are the same address written twice.

None of that is possible while the address is a single string. The work is: split it into components, normalise each component for comparison, and keep enough provenance that you can explain any row later.

This page does it on a real file: the 24,105 businesses on the food hygiene register for Camden, Westminster, Manchester and Leeds, with each address flattened into one comma-separated column the way a customer file arrives. A regular expression finds the postcode and agrees with the separately recorded postcode on 99.97% of rows. The street is harder. Matched against OpenStreetMap's street names for the same four areas, 32.4% of the distinct street lines match verbatim and 44.9% after normalisation — and almost everything that still fails is a parsing problem, not a spelling one.

Quick answer

Normalise for matching, parse for structure, and never overwrite the original:

import re
import unicodedata

ABBREV = {
    "st": "street", "str": "street", "rd": "road", "ave": "avenue",
    "av": "avenue", "ln": "lane", "dr": "drive", "cl": "close",
    "ct": "court", "pl": "place", "sq": "square", "cres": "crescent",
    "n": "north", "s": "south", "e": "east", "w": "west",
}


def normalise(text: str) -> str:
    """Fold everything that varies without changing the place."""
    if not text:
        return ""
    text = unicodedata.normalize("NFKD", text)
    text = "".join(c for c in text if not unicodedata.combining(c))   # drop accents
    text = text.casefold()
    text = re.sub(r"[^\w\s]", " ", text)                              # punctuation to space
    tokens = [ABBREV.get(t, t) for t in text.split()]
    return " ".join(tokens)


print(repr(normalise("Flat 2, 14 High St., CAMDEN")))
print(repr(normalise("Saint-Étienne")))

'flat 2 14 high street camden'
'saint etienne'

Two rows are candidates for the same address when their normalised forms are equal. Everything else in this guide is about the rows where that is not enough.

Two derived columns and one original — the original is the one that must survive.

Profile first, normalise into a new column, take the unambiguous parts

1. Profile the column before you write any rules

Every address file is broken in its own way, and the fastest way to find out how is to count.

import pandas as pd

df = pd.read_csv("customers.csv", dtype=str).fillna("")
addr = df["address"]

print(f"rows              {len(addr):,}")
print(f"distinct verbatim {addr.nunique():,}")
print(f"empty             {(addr.str.strip() == '').sum():,}")
print(f"non-ascii         {addr.str.contains(r'[^\x00-\x7F]').sum():,}")
print(f"has digits        {addr.str.contains(r'\d').sum():,}")
print(f"comma count       {addr.str.count(',').value_counts().head().to_dict()}")
print(f"median length     {addr.str.len().median():.0f}")

rows              24,105
distinct verbatim 19,732
empty             2,563
non-ascii         6
has digits        21,484
comma count       {2: 12406, 3: 7131, 0: 2566, 4: 1726, 1: 276}
median length     39

The comma histogram is the most informative line. Here most rows have two or three commas, which is structure a parser can use, and 2,566 have none: 2,563 of those are empty, mobile traders and home caterers with no public address. A flat histogram would mean free text and a token-based parser.

2. Normalise first, in a new column

Normalisation is cheap, deterministic and reversible only in the sense that you kept the original. Apply it to a new column so both forms exist side by side.

df["address_norm"] = df["address"].map(normalise)
print(f"distinct after normalisation {df['address_norm'].nunique():,}")

distinct after normalisation 19,600

The drop between nunique() before and after is the duplication that was hiding behind formatting. On this register it is small, 114 rows, because each council writes its own addresses consistently; files merged from several sources lose far more.

3. Pull out the unambiguous components first

Parse by subtraction. Take out the things that have a strict format — postcode, unit — then parse what is left, which is smaller and less ambiguous each time. One rule matters more than it looks: the street line is the first comma-separated part that starts with a number, and anything before it is a business or building name. Taking the first part as the street instead turns Regency House, 36-38 Whitworth Street into a street called Regency House with no house number.

import re

UK_POSTCODE = re.compile(r"\b([A-Z]{1,2}\d[A-Z\d]?)\s*(\d[A-Z]{2})\b", re.I)
UNIT = re.compile(r"\b(flat|apt|apartment|unit|suite|room|floor)\.?\s*([\w\-/]+)", re.I)
HOUSE = re.compile(r"^\s*(\d+[a-z]?(?:\s*-\s*\d+[a-z]?)?)\s+(?=\S)", re.I)


def extract(raw: str) -> dict:
    out, text = {"raw": raw}, " ".join(str(raw).split())

    if (m := UK_POSTCODE.search(text)):
        out["postcode"] = f"{m.group(1).upper()} {m.group(2).upper()}"
        text = (text[: m.start()] + " " + text[m.end():]).strip(" ,")

    if (m := UNIT.search(text)):
        out["unit"] = f"{m.group(1).lower()} {m.group(2)}"
        text = (text[: m.start()] + " " + text[m.end():]).strip(" ,")

    parts = [p.strip() for p in text.split(",") if p.strip()]
    # the street line is the first part that starts with a number; anything before it is a name
    first = next((i for i, p in enumerate(parts) if HOUSE.match(p)), 0)
    if first:
        out["name"] = ", ".join(parts[:first])
    parts = parts[first:]
    if parts:
        head = parts[0]
        if (m := HOUSE.match(head)):
            out["house_number"] = m.group(1).replace(" ", "")
            head = head[m.end():]
        out["street_raw"] = head.strip()
        out["street"] = normalise(head)
        if len(parts) > 1:
            out["town"] = parts[-1]
        if len(parts) > 2:
            out["locality"] = parts[-2]
    return out


parsed = df["address"].map(extract)
found = parsed.map(lambda d: (d.get("postcode") or "").replace(" ", ""))
recorded = df["postcode_recorded"].str.replace(" ", "").str.upper()
has = recorded != ""
print(f"postcode found and equal to the recorded one: {(found[has] == recorded[has]).mean():.2%} of {has.sum():,}")
print(f"business or building name split off before the street line: {parsed.map(lambda d: 'name' in d).sum():,}")
print(f"non-empty rows with no house number: {parsed[addr.str.strip() != ''].map(lambda d: 'house_number' not in d).sum():,}")

postcode found and equal to the recorded one: 99.97% of 21,407
business or building name split off before the street line: 1,593
non-empty rows with no house number: 6,394

The postcode rule is the reliable one. The house number is not: 6,394 non-empty rows still have none, and most of those genuinely lack one — a market hall, a station, a sports ground — while others, mostly in Westminster, run the building name and the street together without a comma.

4. Score the parse

A component parser that cannot say how well it did forces everything downstream to assume it did well.

def score(parsed: dict) -> float:
    """Fraction of the input tokens that ended up in a named component."""
    raw_tokens = len(normalise(parsed["raw"]).split())
    used = sum(len(normalise(str(v)).split())
               for k, v in parsed.items() if k not in ("raw", "street_raw"))
    return round(min(1.0, used / max(1, raw_tokens)), 2)

Be clear about what this score measures: how much of the text found a slot, not whether the slot was right. Almost every row here scores 1.0, because whatever the parser cannot place still lands somewhere. The warnings in Example 1 are what catch a name in the street slot.

5. Build the identity key and count what you actually have

def key(p: dict) -> tuple:
    return (
        (p.get("postcode") or "").replace(" ", "").upper(),
        p.get("street", ""),
        (p.get("house_number") or "").lower().lstrip("0"),
        (p.get("unit") or "").lower(),
    )


df["parse_score"] = parsed.map(score)
df["addr_key"] = parsed.map(key)

print(f"{len(df):,} rows -> {df['addr_key'].nunique():,} distinct addresses")
print(f"{(df['parse_score'] < 0.6).sum():,} rows score below 0.6, "
      f"{((df['parse_score'] < 0.6) & (addr.str.strip() != '')).sum()} of them not empty")

24,105 rows -> 19,204 distinct addresses
2,568 rows score below 0.6, 5 of them not empty

This is the number that decides the geocoding budget: 19,204 keys to send rather than 24,105 rows. The rows below 0.6 are almost all the empty addresses, which should not be sent at all.

Each step shortens what is left to guess about, and a step that never fires on a row is itself a warning.

A parser that reports rather than guesses

The data used on this page

The Food Standards Agency's food hygiene rating open data for four local authorities — Camden, Westminster, Manchester and Leeds — with each establishment's address lines and postcode joined into one address column, and the recorded postcode kept beside it as postcode_recorded to check the parser against. The reference street names are every named highway in OpenStreetMap inside the same four council boundaries, fetched through the Overpass API; overpass-api.de refuses connections from here, so the query goes to the maps.mail.ru mirror of the same API. Both change daily, so the setup reads each as saved: the FSA files on 9 October 2026, the street names on 10 October.

import os
import time

import pandas as pd
import requests

AREAS = {"Camden": ("506", 'name="London Borough of Camden"'),
         "Westminster": ("533", 'name="City of Westminster"'),
         "Manchester": ("415", 'wikidata="Q21525592"'),
         "Leeds": ("413", 'name="Leeds"')}
OVERPASS = "https://maps.mail.ru/osm/tools/overpass/api/interpreter"    # overpass-api.de mirror

if not os.path.exists("customers.csv"):
    rows = []
    for area, (code, _) in AREAS.items():
        # the FSA register file as saved on 9 October 2026 (updated daily); live:
        # ratings.food.gov.uk/api/open-data-files/FHRS{code}en-GB.json
        r = requests.get(f"https://data.spatialworkflow.io/fhrs-{code}-2026-10-09.json", timeout=300)
        r.raise_for_status()
        found = r.json()["FHRSEstablishment"]["EstablishmentCollection"]
        for e in (found if isinstance(found, list) else found["EstablishmentDetail"]):
            lines = [e.get(f"AddressLine{i}") or "" for i in (1, 2, 3, 4)] + [e.get("PostCode") or ""]
            rows.append({"id": e["FHRSID"], "area": area, "postcode_recorded": e.get("PostCode") or "",
                         "address": ", ".join(p.strip() for p in lines if p.strip())})
    pd.DataFrame(rows).to_csv("customers.csv", index=False)

if not os.path.exists("osm_streets.csv"):
    # OpenStreetMap is edited daily: these are the names the loop below received on 10 October 2026.
    # Delete these three lines to run the queries live instead.
    r = requests.get("https://data.spatialworkflow.io/osm-street-names-four-councils-2026-10-10.csv", timeout=300)
    r.raise_for_status()
    open("osm_streets.csv", "wb").write(r.content)
if not os.path.exists("osm_streets.csv"):
    names = []
    for area, (_, tag) in AREAS.items():
        q = (f'[out:csv(name;false)][timeout:300];area["boundary"="administrative"]["admin_level"="8"][{tag}]->.a;'
             'way(area.a)["highway"]["name"];out;')
        for attempt in range(5):                          # the mirror sometimes answers 504
            r = requests.post(OVERPASS, data={"data": q}, timeout=600)
            if r.ok:
                break
            time.sleep(30)
        r.raise_for_status()
        names += [{"area": area, "name": n} for n in r.text.splitlines() if n.strip()]
    pd.DataFrame(names).drop_duplicates().to_csv("osm_streets.csv", index=False)
print(pd.read_csv("customers.csv").area.value_counts().to_dict(),
      len(pd.read_csv("osm_streets.csv")), "distinct street names by area")

{'Leeds': 7460, 'Manchester': 6599, 'Westminster': 5753, 'Camden': 4293} 23719 distinct street names by area

Example 1 — a parser that reports rather than guesses

from dataclasses import dataclass, field


@dataclass
class ParsedAddress:
    raw: str
    country: str = "GB"
    name: str | None = None
    unit: str | None = None
    house_number: str | None = None
    street: str | None = None
    locality: str | None = None
    town: str | None = None
    postcode: str | None = None
    score: float = 0.0
    warnings: list[str] = field(default_factory=list)

    def check(self) -> "ParsedAddress":
        if not self.raw.strip():
            self.warnings.append("empty")
            return self
        if self.name:
            self.warnings.append(f"name before the street line: {self.name!r}")
        if self.house_number and not self.street:
            self.warnings.append("house number with no street")
        if not self.house_number and self.street:
            self.warnings.append("no house number — a building or place name?")
        if not self.postcode and not self.town:
            self.warnings.append("no locality — cannot disambiguate the street")
        if re.search(r"\b(po box|p\.o\. box|freepost)\b", self.raw, re.I):
            self.warnings.append("PO box — not a physical location")
        if self.score < 0.6:
            self.warnings.append(f"low parse score {self.score}")
        return self


def parse(raw: str, country: str = "GB") -> ParsedAddress:
    d = extract(raw)
    return ParsedAddress(
        raw=raw, country=country, name=d.get("name"), unit=d.get("unit"),
        house_number=d.get("house_number"), street=d.get("street"),
        locality=d.get("locality"), town=d.get("town"),
        postcode=d.get("postcode"), score=score(d),
    ).check()


results = df["address"].map(parse)
warned = results.map(lambda p: [w.split(" —")[0].split(":")[0] for w in p.warnings])
print(pd.Series([w for ws in warned for w in ws]).value_counts().to_string())
for raw in ["Regency House, 36-38 Whitworth Street, Manchester, M1 3NR",
            "COMMUNICATIONS HOUSE 48 LEICESTER SQUARE, LONDON, WC2H 7LT"]:
    print(raw, "->", parse(raw).warnings)

no house number                6393
empty                          2563
name before the street line    1593
no locality                       3
PO box                            2
low parse score 0.56              2
low parse score 0.54              1
low parse score 0.58              1
low parse score 0.57              1
Regency House, 36-38 Whitworth Street, Manchester, M1 3NR -> ["name before the street line: 'Regency House'"]
COMMUNICATIONS HOUSE 48 LEICESTER SQUARE, LONDON, WC2H 7LT -> ['no house number — a building or place name?']

Two of the register's rows are Freepost or PO box addresses, which are not places at all. The Leicester Square row shows the limit of comma-based parsing: with no comma between the building and the street, the whole line becomes the street and the warning is the only sign.

Example 2 — normalising a whole file efficiently

import time
import pandas as pd


def normalise_column(s: pd.Series) -> pd.Series:
    """Vectorised where it can be, mapped where it cannot.

    The accent fold is the expensive part, so it runs once per distinct
    value rather than once per row — address files repeat heavily.
    """
    uniq = pd.Series(s.dropna().unique())
    lookup = dict(zip(uniq, uniq.map(normalise)))
    return s.map(lookup).fillna("")


for label, run in [("every row", lambda: df["address"].map(normalise)),
                   ("distinct values", lambda: normalise_column(df["address"]))]:
    start = time.perf_counter()
    for _ in range(10):
        run()
    print(f"{label:16} {(time.perf_counter() - start) / 10 * 1000:5.0f} ms")

every row           68 ms
distinct values     83 ms

Mapping over the distinct values instead of the rows is a common trick, and here it loses: 19,732 of the 24,105 strings are distinct, so there is little repetition to save and unique() costs more than it saves. It pays when a file repeats heavily — a transaction table with a few thousand addresses behind a million rows. Measure before adopting it.

Westminster writes its addresses in capitals, so folding case is the step that pays.

Example 3 — comparing normalisation strategies on your own data

import difflib
import random
import time
import unicodedata


def fold_accents(s):
    s = unicodedata.normalize("NFKD", s)
    return "".join(c for c in s if not unicodedata.combining(c))


STRATEGIES = {
    "verbatim":   lambda s: s,
    "lowercase":  lambda s: s.casefold(),
    "+accents":   lambda s: fold_accents(s.casefold()),
    "+punct":     lambda s: " ".join(re.sub(r"[^\w\s]", " ", fold_accents(s.casefold())).split()),
    "+abbrev":    normalise,
}


def compare_strategies(queries, reference):
    """How much does each normalisation step actually buy you? Matched within each area."""
    for label, fn in STRATEGIES.items():
        hits = 0
        for area, q in queries.groupby("area"):
            index = {fn(n) for n in reference.loc[reference.area == area, "name"]}
            hits += q.street_raw.map(fn).isin(index).sum()
        print(f"{label:11s} {100 * hits / len(queries):5.1f}%")


streets = pd.read_csv("osm_streets.csv")
queries = (pd.DataFrame({"area": df["area"], "street_raw": parsed.map(lambda d: d.get("street_raw", ""))})
           .query("street_raw != ''").drop_duplicates())
print(f"{len(queries):,} distinct street lines, {len(streets):,} OpenStreetMap street names")
compare_strategies(queries, streets)

index = {area: {normalise(n): n for n in g.name} for area, g in streets.groupby("area")}
residue = [(a, q) for a, q in queries.itertuples(index=False) if normalise(q) not in index[a]]
random.seed(4)
sample = random.sample(residue, 200)
print(f"\n{len(residue):,} unmatched, for example:", "; ".join(q for _, q in sample[:12]))

start = time.perf_counter()
fuzzy = [difflib.get_close_matches(normalise(q), list(index[a]), n=1, cutoff=0.85) for a, q in sample]
per_fuzzy = (time.perf_counter() - start) / len(sample)
keys = [normalise(q) for _, q in sample]
start = time.perf_counter()
for _ in range(1000):
    for (a, _), k in zip(sample, keys):
        k in index[a]
per_hash = (time.perf_counter() - start) / (1000 * len(sample))
print(f"fuzzy: {per_fuzzy * 1000:.0f} ms a query, candidate for {sum(map(bool, fuzzy))} of {len(sample)}; "
      f"dictionary: {per_hash * 1e6:.2f} µs a query")
for (a, q), m in list(zip(sample, fuzzy))[:40]:
    if m:
        print(f"  {q!r} -> {index[a][m[0]]!r}")
shared = streets.assign(k=streets.name.map(normalise)).groupby("k").area.nunique()
print(f"normalised street names found in more than one of the four areas: {(shared > 1).sum():,} of {len(shared):,}")

7,115 distinct street lines, 23,719 OpenStreetMap street names
verbatim     32.4%
lowercase    44.6%
+accents     44.6%
+punct       44.8%
+abbrev      44.9%

3,922 unmatched, for example: THE ESPRESSO BAR; GROUND   95 HARLEY STREET; West Hampstead Community Hall; Chapel Wood; BASEMENT AND GROUND   NORTH AUDLEY STREET; Authentic City Church; TATE BRITAIN MILLBANK; - 58; Hampstead Lane; Bloomsbury Market; NORTH WESTMINSTER COLLEGE 129 ELGIN AVENUE; The Galleon Hotel And Leisure Club
fuzzy: 3 ms a query, candidate for 5 of 200; dictionary: 0.07 µs a query
  'Hampstead Lane' -> 'Hampstead Gate'
normalised street names found in more than one of the four areas: 1,087 of 22,498

Case folding is the step that matters on this data, because Westminster writes addresses in capitals and OpenStreetMap does not. Punctuation adds 0.2 points and abbreviation expansion 0.1. The ceiling of about 45% is set by the parser: the unmatched lines are building names, floors (GROUND, THIRD), a town left on the end of the street, and building-and-street runs with no comma. Fuzzy matching does not reach those, and it costs about 3 milliseconds a query against well under a microsecond for the dictionary.

Normalisation is a hash, fuzzy matching a search

Why normalisation beats fuzzy matching for the bulk of the work

Normalisation is a hash: it maps many spellings to one key and then does a dictionary lookup. Fuzzy matching is a search: it compares the query against candidates and scores each one.

The cost difference is enormous. On the four councils' street names, a dictionary lookup took 0.07 microseconds and difflib.get_close_matches at a 0.85 cutoff about 3 milliseconds — tens of thousands of times more. On this file it also found a candidate for only 5 of 200 unmatched lines, and a candidate is not a match: Hampstead Lane came back as Hampstead Gate, a different street, which needs a person to judge. Fuzzy matching earns its cost on typos, and most of what normalisation leaves behind here is not a typo.

Why abbreviation expansion is worth so little on its own

In the strategy comparison, expanding abbreviations moved the match rate from 44.8% to 44.9%. Case folding moved it 12.2 points. Abbreviations are visible, so people over-invest in them; case, accents and stray whitespace are invisible, so they go unfixed.

The other reason to keep the abbreviation table small: every entry is a guess that can be wrong. St is Street or Saint — St John's Wood Road becomes street john s wood road under this table — and N is North or a house number suffix. A short table of unambiguous mappings is worth more than a long one that introduces errors.

Why parsing by subtraction works

Free-text parsing is hard because the parser has to decide what everything is at once. Removing the postcode first turns an ambiguous token sequence into a shorter, more constrained one, and the postcode is trivially identifiable by format — which is why it was right on all but a handful of 21,407 rows.

Order the extraction by how strictly formatted each component is: postcode, then country, then unit keywords, then the numbered line and its house number, then whatever is left. Each removal makes the next step easier — and each one that fails to fire is itself a signal about the row.

Why the identity key excludes the town

Two records are the same address when they name the same front door. The town is context that helps a geocoder find the street; it is not part of the door's identity, and including it splits 14 High Street, Camden from 14 High Street, London. It matters because street names repeat: 1,087 normalised street names occur in more than one of these four council areas.

The postcode is the exception: it is fine-grained enough (in the UK, about fifteen addresses) to be part of identity rather than context. In countries where the postal code covers thousands of addresses, it is context too, and the street name has to carry the disambiguation.

Normalise everything, fuzzy-match only what normalisation could not place.

Raw columns, casefold and Turkish I

Never parse in place. Keep raw forever; parsers improve and you will want to re-run.
Use casefold(), not lower(). It handles German ß and other multi-character folds correctly.
Watch Turkish I. Locale-aware casing turns I into ı; if your data mixes locales, normalise with casefold() and accept the imperfection.
normalize("NFKD") also folds ligatures and full-width characters, which is usually what you want for matching and never what you want for display.
Leading zeros. 007 and 7 are the same house number; strip them in the key only.
Do not normalise the postcode away. Uppercase it and remove spaces, but keep the characters — it is a checkable code, not free text.
Names in the first line are common in business files: 1,593 rows here put one before the numbered street line.
Test the parser on the rows it warns about, not on the examples you wrote it from.

Internal links

The address data model: why an address is not a string — the schema this parser fills in
Address matching explained: why exact string equality fails — what to do with the normalised keys
How to deduplicate an address list in Python — using the identity key at scale
How to match addresses against a reference file — joining to an authoritative gazetteer
How to batch geocode thousands of addresses in Python — the step this one saves money on
How to standardise attribute values — the same idea for non-address columns
Fixing addresses that break on accents and encoding — when the text is damaged before you get it
How to clean an address list before geocoding — the wider cleaning checklist

FAQ

Should I write my own address parser or use a library?

Write rules for a single-country file — they are debuggable and fast. Use a statistical parser such as libpostal when the input is genuinely international and unstructured.

How much does normalisation actually improve matching?

On the four councils' food hygiene addresses, matched against OpenStreetMap street names, from 32.4% of distinct street lines verbatim to 44.9% normalised. Case folding did almost all of it; abbreviation expansion added 0.1 points.

Is fuzzy matching worth it?

On typos, yes; on this file, little. It found a candidate for 5 of 200 unmatched street lines, at about 3 milliseconds a query against 0.07 microseconds for a dictionary lookup, because most unmatched lines were parsing failures rather than misspellings.

What do I do with rows the parser cannot handle?

Route them by warning. A name before the street, no house number or no locality each send a row to review; empty addresses should not reach a geocoder at all.

Does normalisation ever cause wrong matches?

Yes. Normalising makes different streets look alike and does nothing about names that genuinely repeat: 1,087 normalised street names occur in more than one of the four council areas, which is why a match has to be within the right area.