How to suppress small counts in a spatial table in Python

Problem statement

You have a table of counts by area and category, and some of the counts are small enough to identify someone. Blanking them is the obvious move and, done naively, it does not work: a hidden value in a row whose total is published is a subtraction, not a secret.

A correct suppression pass has three stages — primary suppression of cells below the threshold, complementary suppression of enough extra cells that nothing can be derived from a margin, and a decision about what to publish in place of the holes. This guide implements all three on a real table: 22,293 central London crime records for the first quarter of 2025, aggregated to 180 cells of 250 m by 14 crime categories.

Quick answer

Suppress the small cells, then iterate until no row or column has exactly one hole:

import numpy as np

MIN = 5
values = table.values
nonempty = values > 0
sup = nonempty & (values < MIN)                      # primary suppression
primary = sup.sum()

passes = 0
while True:
    passes += 1
    changed = False
    for i in range(values.shape[0]):                 # rows
        if sup[i].sum() == 1:
            cand = np.flatnonzero(nonempty[i] & ~sup[i])
            if len(cand):
                sup[i, cand[values[i, cand].argmin()]] = True
                changed = True
    for j in range(values.shape[1]):                 # columns
        if sup[:, j].sum() == 1:
            cand = np.flatnonzero(nonempty[:, j] & ~sup[:, j])
            if len(cand):
                sup[cand[values[cand, j].argmin()], j] = True
                changed = True
    if not changed:
        break
print(f"{passes} passes: {primary} primary suppressions became {sup.sum()}, hiding {values[sup].sum():,} of "
      f"{values.sum():,} crimes ({values[sup].sum() / values.sum():.1%}) across "
      f"{sup.sum() / nonempty.sum():.1%} of the non-empty cells")

2 passes: 960 primary suppressions became 964, hiding 2,016 of 22,293 crimes (9.0%) across 56.1% of the non-empty cells

On the London table this stabilised after two passes: 960 primary suppressions became 964, hiding 2,016 of 22,293 crimes (9.0%) across 56.1% of the non-empty cells.

Step-by-step solution

1. Build the table at the finest unit the release exposes

If users can filter by month and by category, the table is cell × month × category. Suppressing a coarser table protects nothing that the fine one gives away.

2. Count subjects, not rows

Deduplicate to subjects before the crosstab if one person can appear several times. A cell of five records from two people is a cell of two.

3. Apply primary suppression

Everything strictly below the threshold and strictly above zero. Whether to suppress zeros as well is a separate decision — see step 6.

4. Iterate complementary suppression to a fixed point

A row or column with exactly one hole gives that hole away when its margin is published. Suppressing the smallest remaining value costs the least information. The loop terminates quickly in practice; on the London table, two passes and four extra cells.

5. Handle the rows that cannot be fixed

Some rows have only one non-empty cell, so there is no second value to hide. None of the 180 London rows was, because every 250 m cell of central London logged several kinds of crime; at 100 m or in a sparse area it is common. The only honest options are to suppress the row total as well, or to merge the row into a neighbouring area.

6. Decide what a blank means, and say so

A blank is ambiguous between "zero" and "suppressed", and that ambiguity is useful — but only if documented. The usual choices are a sentinel (-1, "<5"), a separate flag column, or publishing a band. A flag column is easiest to consume.

7. Weigh the threshold against what it costs

The same table under three thresholds, after the complementary passes:

Between thresholds 3 and 10 the cells blanked roughly double; the records hidden more than quadruple, because the distribution is concentrated in the single digits.

8. Consider rounding instead

Controlled rounding to a base of 3 or 5 publishes a number everywhere. Users generally prefer a table of approximate numbers to a table half full of holes, and it removes the complementary-suppression problem entirely.

9. Check the published margins agree

After suppression, the sum of the published values plus the suppressed mass must equal the published total, or users will find the discrepancy and reconstruct it.

Code examples

The data used on this page

Street-level crime from data.police.uk for January to March 2025, for a box over Westminster, Holborn and Bloomsbury (0.16°W to 0.10°W, 51.505°N to 51.530°N), fetched in tiles because the API returns at most 10,000 crimes a request. It is saved as london_2025q1.gpkg, and crossed by 250 m British National Grid cell and the 14 police.uk crime categories.

import os

import geopandas as gpd
import numpy as np
import pandas as pd
import requests

WEST, SOUTH, EAST, NORTH = -0.16, 51.505, -0.10, 51.530      # Westminster, Holborn, Bloomsbury
if not os.path.exists("london_crime_2025q1.csv"):
    rows = []
    for month in ("2025-01", "2025-02", "2025-03"):
        for x0, x1 in zip(np.linspace(WEST, EAST, 4)[:-1], np.linspace(WEST, EAST, 4)[1:]):
            for y0, y1 in zip(np.linspace(SOUTH, NORTH, 3)[:-1], np.linspace(SOUTH, NORTH, 3)[1:]):
                reply = requests.post("https://data.police.uk/api/crimes-street/all-crime", timeout=120,
                                      data={"poly": f"{y0},{x0}:{y0},{x1}:{y1},{x1}:{y1},{x0}", "date": month})
                reply.raise_for_status()                    # a 503 means over 10,000 crimes: split further
                rows += [{"id": c["id"], "month": c["month"], "category": c["category"],
                          "lon": float(c["location"]["longitude"]), "lat": float(c["location"]["latitude"]),
                          "street_id": c["location"]["street"]["id"]} for c in reply.json()]
    pd.DataFrame(rows).drop_duplicates("id").to_csv("london_crime_2025q1.csv", index=False)
crimes = pd.read_csv("london_crime_2025q1.csv")
crimes = crimes[crimes.lon.between(WEST, EAST) & crimes.lat.between(SOUTH, NORTH)]
crimes = gpd.GeoDataFrame(crimes, geometry=gpd.points_from_xy(crimes.lon, crimes.lat), crs=4326).to_crs(27700)
print(f"{len(crimes):,} crimes, {crimes.category.nunique()} categories, {crimes.street_id.nunique():,} snap points")
crimes.to_file("london_2025q1.gpkg", driver="GPKG")
cell = (crimes.geometry.x // 250).astype(int).astype(str) + "_" + (crimes.geometry.y // 250).astype(int).astype(str)
table = pd.crosstab(cell, crimes["category"])
print(f"table {table.shape}")

22,293 crimes, 14 categories, 1,249 snap points
table (180, 14)

Example 1 — the table and the primary pass

import geopandas as gpd, pandas as pd

crimes = gpd.read_file("london_2025q1.gpkg").to_crs(27700)
cell = (crimes.geometry.x // 250).astype(int).astype(str) + "_" + \
       (crimes.geometry.y // 250).astype(int).astype(str)
table = pd.crosstab(cell, crimes["category"])

MIN = 5
nonempty = table.values > 0
primary = nonempty & (table.values < MIN)
print(f"table {table.shape}, non-empty {nonempty.sum():,}, "
      f"primary suppressions {primary.sum():,} "
      f"({primary.sum()/nonempty.sum():.1%}) hiding {table.values[primary].sum():,} records")

table (180, 14), non-empty 1,719, primary suppressions 960 (55.8%) hiding 1,992 records

Example 2 — complementary suppression as a function

import numpy as np

def suppress(table, min_count=5, suppress_zeros=False):
    v = table.values
    nonempty = v > 0 if not suppress_zeros else np.ones_like(v, bool)
    sup = nonempty & (v < min_count)

    for _ in range(100):
        changed = False
        for axis in (0, 1):
            s = sup if axis == 1 else sup.T
            ne = nonempty if axis == 1 else nonempty.T
            vals = v if axis == 1 else v.T
            for i in range(s.shape[0]):
                if s[i].sum() == 1:
                    cand = np.flatnonzero(ne[i] & ~s[i])
                    if len(cand):
                        s[i, cand[vals[i, cand].argmin()]] = True
                        changed = True
        if not changed:
            break

    out = table.mask(sup)
    return out, sup

published, mask = suppress(table, 5)
print(f"suppressed {mask.sum():,} of {(table.values > 0).sum():,} non-empty "
      f"({mask.sum()/(table.values > 0).sum():.1%}), "
      f"hiding {table.values[mask].sum():,} records "
      f"({table.values[mask].sum()/table.values.sum():.1%})")

suppressed 964 of 1,719 non-empty (56.1%), hiding 2,016 records (9.0%)

Four extra cells over the primary pass — complementary suppression is cheap here because the table is wide and the small values are spread out.

Example 3 — publishing the result unambiguously

long = (published.stack()                     # pandas 3 keeps the NaN cells; pandas 2 needs future_stack=True
        .rename("count").reset_index()
        .rename(columns={"row_0": "cell"}))
long["flag"] = np.where(long["count"].isna(), "suppressed",
                        np.where(long["count"] == 0, "zero", "published"))
long["count"] = long["count"].fillna(-1).astype(int)

print(long["flag"].value_counts().to_string())

flag
suppressed    964
zero          801
published     755

Example 4 — thresholds, and the rows that cannot be protected

one = (table.values > 0).sum(axis=1) == 1
print(f"rows with a single non-empty cell: {one.sum()} of {len(table)}")
for m in (3, 5, 10):
    _, mk = suppress(table, m)
    ne = (table.values > 0).sum()
    hidden = table.values[mk].sum()
    print(f"threshold {m:2d}: {mk.sum():,} cells ({mk.sum() / ne:.1%} of non-empty), "
          f"{hidden:,} records ({hidden / table.values.sum():.1%})")

rows with a single non-empty cell: 0 of 180
threshold  3: 660 cells (38.4% of non-empty), 960 records (4.3%)
threshold  5: 964 cells (56.1% of non-empty), 2,016 records (9.0%)
threshold 10: 1,269 cells (73.8% of non-empty), 4,040 records (18.1%)

Ship the flag. Without it, every user writes their own guess about what a blank means, and half of them guess zero.

Explanation

Why the complementary pass is mandatory

Suppression removes a value from the page, not from the arithmetic. With four of five categories published and the row total published, the fifth is a subtraction. The loop enforces the minimal property that makes the subtraction underdetermined: at least two unknowns in every equation.

Why "at least two holes" is a floor, not a guarantee

Two unknowns in a row is underdetermined only if the same two are not each determined by their own columns. Statistical agencies solve this properly as a linear programme that gives every suppressed cell a feasible interval wider than a protection level. The iterative heuristic here is adequate for open data and inadequate for a release where the attacker will spend real effort; if the table has many margins and subtotals, use a purpose-built cell-suppression solver.

Why raising the threshold is not free

Going from 5 to 10 on this table changes a small number of hidden records into a large number of hidden cells, because most of the distribution sits in the single digits. Look at both costs — cells lost and records hidden — before choosing.

Why rounding often wins

Controlled rounding turns every cell into an approximate number instead of a hole. Nothing has to be suppressed, the margins can be made consistent, and the user gets a usable table. Its weakness is that a rounded value still bounds the truth, so it needs a base large enough that the bound is not disclosive.

Edge cases or notes

• Suppress subtotals too. A subtotal is a margin like any other.
• Time series need a joint decision. A cell suppressed once and published later is disclosed by the annual total.
• Percentages and rates leak. 100% of a cell of 3 publishes the 3.
• Zeros may need suppressing. In a small known population, a zero can be as revealing as a one.
• Rows with a single non-empty cell cannot be fixed by complementary suppression. Suppress the total or merge the row.
• Keep the mask, not just the masked table. You need it next release to stay consistent.
• Do not suppress after adding noise. Mixing differential privacy with a data-dependent suppression breaks the guarantee.
• Publish the rule. Users must be able to tell a blank from a zero without asking.

Internal links

• Aggregation and suppression rules explained — the threshold rules and why the cross-product matters
• Suppressed cells can be recovered from the totals — the failure mode this prevents
• How to aggregate points to units that meet a minimum count — avoiding suppression by changing the geography
• How to add differentially private noise to spatial counts — the alternative to holes
• Spatial k-anonymity explained — where the threshold comes from
• Small area rates explained — why small denominators are also a disclosure channel
• How to normalise census counts into rates in Python — the denominators that leak
• Margins of error explained — published uncertainty around suppressed cells

FAQ

What counts as a small count?

Anything below the threshold you publish — commonly 3, 5 or 10. The threshold applies to subjects, not records.

Why is suppressing the small cells not enough?

Because a single hole in a row with a published total can be recovered by subtraction. Every row and column needs zero or at least two holes.

How many extra cells does complementary suppression cost?

Usually few. On the London table it added four cells to 960, taking the total hidden from 1,992 to 2,016 records.

What should I put in a suppressed cell?

A sentinel plus a flag column, so users can tell a suppressed cell from a true zero. A bare blank is ambiguous and will be read as zero.

Should I round instead of suppressing?

Often yes. Controlled rounding to a base of 3 or 5 publishes a number everywhere and avoids the complementary-suppression problem.

What if a row has only one non-empty cell?

There is nothing to pair it with. Suppress the row total as well, or merge the row into an adjacent area.