🔧 Fix Privacy & sensitive locations Census & demographics Geoprivacy & Anonymisation · step 18 Notebook By · Updated

Suppressed cells can be recovered from the totals

On this page

Problem statement

The table was suppressed correctly by the rule everyone uses: any cell below five is blanked. Then a user noticed that one row published four of its five categories and its total, and subtracted.

Suppression hides values from the page, not from the arithmetic. A blanked cell in a row or column with exactly one hole and a published margin has a single solution; a table with several holes can still be solved if the holes are arranged badly. On a real table — 180 cells of 250 m central London crime by 14 categories, from data.police.uk — 4 of the 180 rows had exactly one suppressed value and were therefore fully recoverable.

Quick answer

Find the recoverable cells, then suppress a second value in each affected row and column until none remain:

import numpy as np

v = table.values
nonempty = v > 0
sup = nonempty & (v < 5)                       # primary suppression

while True:
    changed = False
    for i in range(v.shape[0]):
        if sup[i].sum() == 1:
            cand = np.flatnonzero(nonempty[i] & ~sup[i])
            if len(cand):
                sup[i, cand[v[i, cand].argmin()]] = True
                changed = True
    for j in range(v.shape[1]):
        if sup[:, j].sum() == 1:
            cand = np.flatnonzero(nonempty[:, j] & ~sup[:, j])
            if len(cand):
                sup[cand[v[cand, j].argmin()], j] = True
                changed = True
    if not changed:
        break
primary = nonempty & (v < 5)
print(f"{primary.sum():,} primary suppressions hiding {v[primary].sum():,} records -> "
      f"{sup.sum():,} after the repair, hiding {v[sup].sum():,}")
960 primary suppressions hiding 1,992 records -> 964 after the repair, hiding 2,016

On the London table this cost four extra cells: 960 primary suppressions became 964, hiding 2,016 records instead of 1,992.

Four ways a suppressed cell is recoverable, with the fix for each. how a hole stops being a holechannelfixone hole in a row with a totalsuppress a second value in the rowone hole in a column with a totalsuppress a second value in thecolumnsuppressed one month, publishedthe nextstore and reuse the maskwards published under a districttotaldecide across both geographiesThe last two are the ones that get missed spatialworkflow.io
The row total is the obvious channel; the time series and the second geography are the ones that get missed.

Every margin, single holes, then iterate

1. Enumerate every published margin

Row totals, column totals, the grand total, subtotals, "all categories" rows, the map legend's maximum, and any figure quoted in the accompanying text. Each is an equation.

2. Find rows and columns with exactly one hole

sup = (table > 0) & (table < 5)
print("rows with one hole:", int((sup.sum(axis=1) == 1).sum()))
print("cols with one hole:", int((sup.sum(axis=0) == 1).sum()))
rows with one hole: 4
cols with one hole: 0

Four recoverable cells out of 960 suppressions. Small, and completely defeating the purpose for those four — Example 1 recovers every one of them exactly.

3. Suppress the cheapest additional cell in each

Choosing the smallest remaining value minimises the information lost, because a small value is the least useful published number in the row.

4. Iterate — one fix creates another

Suppressing a second cell in a row can leave that cell's column with exactly one hole. Loop until a pass makes no change.

5. Handle rows that cannot be repaired

A row whose only non-empty cell is suppressed has no second value to hide. None of the 180 rows here was in that position, but sparse tables have them. The only correct responses are to suppress the row total as well, or to merge the row into a neighbouring area. Leaving it is publishing the value.

6. Check the time dimension

A cell suppressed in one month and published in another, with a published annual total, is recoverable by subtraction across the series. Suppression decisions must be made jointly over every release, not per release.

7. Check the second geography

If the same population is published at two nested geographies — wards and districts — the difference between a district and the sum of its published wards bounds the suppressed wards. Overlapping non-nested geographies are worse, because they produce a system of equations.

8. Consider rounding instead of holes

Controlled rounding publishes a number in every cell and removes the arithmetic entirely. It is usually the better answer for open data, and it is what several national statistics offices now do.

The iteration that removes recoverable cells from a suppressed table. iterate to a fixed point1blank every cell below the threshold643 cells2find rows with exactly one hole6 of 1303blank the smallest remaining valuecheapest information4repeat until nothing changes2 passes, +3 cellsCentral London crime, 250 m cells × 14 categories spatialworkflow.io
Four knowns and a total make the fifth value arithmetic, not a secret.

An attacker's solver and its bounds

The data used on this page

Street-level crime from data.police.uk for January to March 2025, for a box over Westminster, Holborn and Bloomsbury (0.16°W to 0.10°W, 51.505°N to 51.530°N), fetched in tiles because the API returns at most 10,000 crimes a request. It is crossed by 250 m British National Grid cell and the 14 police.uk crime categories.

import os
import time

import geopandas as gpd
import numpy as np
import pandas as pd
import requests

WEST, SOUTH, EAST, NORTH = -0.16, 51.505, -0.10, 51.530      # Westminster, Holborn, Bloomsbury
if not os.path.exists("london_crime_2025q1.csv"):
    rows = []
    for month in ("2025-01", "2025-02", "2025-03"):
        for x0, x1 in zip(np.linspace(WEST, EAST, 4)[:-1], np.linspace(WEST, EAST, 4)[1:]):
            for y0, y1 in zip(np.linspace(SOUTH, NORTH, 3)[:-1], np.linspace(SOUTH, NORTH, 3)[1:]):
                for attempt in range(5):                        # the API answers bursts with 429
                    reply = requests.post("https://data.police.uk/api/crimes-street/all-crime", timeout=120,
                                          data={"poly": f"{y0},{x0}:{y0},{x1}:{y1},{x1}:{y1},{x0}", "date": month})
                    if reply.status_code != 429:
                        break
                    time.sleep(2 ** attempt)
                reply.raise_for_status()                    # a 503 means over 10,000 crimes: split further
                rows += [{"id": c["id"], "month": c["month"], "category": c["category"],
                          "lon": float(c["location"]["longitude"]), "lat": float(c["location"]["latitude"]),
                          "street_id": c["location"]["street"]["id"]} for c in reply.json()]
    pd.DataFrame(rows).drop_duplicates("id").to_csv("london_crime_2025q1.csv", index=False)
crimes = pd.read_csv("london_crime_2025q1.csv")
crimes = crimes[crimes.lon.between(WEST, EAST) & crimes.lat.between(SOUTH, NORTH)]
crimes = gpd.GeoDataFrame(crimes, geometry=gpd.points_from_xy(crimes.lon, crimes.lat), crs=4326).to_crs(27700)
print(f"{len(crimes):,} crimes, {crimes.category.nunique()} categories, {crimes.street_id.nunique():,} snap points")
cell = (crimes.geometry.x // 250).astype(int).astype(str) + "_" + (crimes.geometry.y // 250).astype(int).astype(str)
table = pd.crosstab(cell, crimes["category"])
print(f"table {table.shape}")
22,293 crimes, 14 categories, 1,249 snap points
table (180, 14)

Example 1 — an attacker's solver

import numpy as np

def recover(published, row_totals, col_totals):
    """Return cells that are determined exactly by the published margins."""
    known = ~np.isnan(published)
    solved = {}
    changed = True
    while changed:
        changed = False
        for i, total in enumerate(row_totals):
            missing = np.flatnonzero(~known[i])
            if len(missing) == 1:
                j = missing[0]
                published[i, j] = total - np.nansum(published[i])
                known[i, j] = True
                solved[(i, j)] = published[i, j]
                changed = True
        for j, total in enumerate(col_totals):
            missing = np.flatnonzero(~known[:, j])
            if len(missing) == 1:
                i = missing[0]
                published[i, j] = total - np.nansum(published[:, j])
                known[i, j] = True
                solved[(i, j)] = published[i, j]
                changed = True
    return solved

v = table.values.astype(float)
primary = (v > 0) & (v < 5)
published = np.where(primary, np.nan, v)                 # the release with primary suppression only
row_totals, col_totals = v.sum(axis=1), v.sum(axis=0)
solved = recover(published.copy(), row_totals, col_totals)
print(f"{len(solved)} suppressed cells recovered exactly")
print("all correct:", all(value == v[cell] for cell, value in solved.items()))
4 suppressed cells recovered exactly
all correct: True

Run this against your own release. If it returns anything, the suppression is incomplete.

Example 2 — bounds, not just exact solutions

from scipy.optimize import linprog

def feasible_range(published, row_totals, col_totals, target):
    """Minimum and maximum a suppressed cell can take, given the margins."""
    idx = [(i, j) for i in range(published.shape[0])
           for j in range(published.shape[1]) if np.isnan(published[i, j])]
    n = len(idx)
    A, b = [], []
    for i, total in enumerate(row_totals):
        row = [1 if a == i else 0 for a, _ in idx]
        if any(row):
            A.append(row); b.append(total - np.nansum(published[i]))
    for j, total in enumerate(col_totals):
        col = [1 if b_ == j else 0 for _, b_ in idx]
        if any(col):
            A.append(col); b.append(total - np.nansum(published[:, j]))
    c = np.zeros(n); c[idx.index(target)] = 1
    lo = linprog(c, A_eq=A, b_eq=b, bounds=[(0, None)] * n)
    hi = linprog(-c, A_eq=A, b_eq=b, bounds=[(0, None)] * n)
    return lo.fun, -hi.fun


leaked = next(iter(solved))
print(f"cell {tuple(int(i) for i in leaked)}, true value {v[leaked]:.0f}: feasible range "
      f"{feasible_range(published.copy(), row_totals, col_totals, leaked)}")
cell (31, 9), true value 2: feasible range (2.0, 2.0)

A cell whose feasible range is [2, 2] is published. A cell whose range is [0, 4] is protected. Statistical agencies size the suppression pattern by requiring every range to be wider than a protection level — which is what the heuristic loop approximates.

Example 3 — what the repair costs

def suppress(table, min_count=5):
    """The quick answer's loop as a function: primary plus complementary suppression."""
    v = table.values
    nonempty = v > 0
    sup = nonempty & (v < min_count)
    changed = True
    while changed:
        changed = False
        for s, ne, vals in ((sup, nonempty, v), (sup.T, nonempty.T, v.T)):
            for i in range(s.shape[0]):
                if s[i].sum() == 1:
                    cand = np.flatnonzero(ne[i] & ~s[i])
                    if len(cand):
                        s[i, cand[vals[i, cand].argmin()]] = True
                        changed = True
    return table.mask(sup), sup


before = ((table > 0) & (table < 5)).values
after = suppress(table, 5)[1]

print(f"primary:      {before.sum():,} cells, {table.values[before].sum():,} records")
print(f"complementary:{after.sum():,} cells, {table.values[after].sum():,} records")
print(f"extra cost:   {after.sum() - before.sum()} cells, "
      f"{table.values[after].sum() - table.values[before].sum()} records")
single = ((table.values > 0).sum(axis=1) == 1) & (before.sum(axis=1) == 1)
print(f"rows whose only non-empty cell is suppressed, so no second cell exists: {int(single.sum())}")
primary:      960 cells, 1,992 records
complementary:964 cells, 2,016 records
extra cost:   4 cells, 24 records
rows whose only non-empty cell is suppressed, so no second cell exists: 0

Four cells and 24 records. The fix is nearly free on a wide table, which is why there is no excuse for skipping it.

Why the two-holes rule is not a guarantee

Why the heuristic is not a guarantee

Requiring at least two holes per row and column makes each single equation underdetermined. It does not prove the system is: a cell with two holes in its row can still be pinned if the other hole is fixed by its own column. Proper cell suppression solves a linear programme that gives every suppressed cell a feasible interval of a chosen width. For an open-data table the heuristic is usually adequate; for a release under attack, use a solver and check the intervals.

Why time series are the commonest real failure

Nobody re-derives the suppression pattern across twelve monthly releases and an annual total, because the twelve releases are produced by twelve pipeline runs that know nothing about each other. The suppression state has to be stored and reused, exactly like a mask.

Why nested geographies leak

Publishing wards and districts publishes a set of equations linking them. If a district has five wards and one ward is suppressed, the district total minus the four published wards is the suppressed value. Two geographies are two chances to get the pattern wrong.

Why rounding sidesteps the whole problem

With controlled rounding there are no holes, so there are no equations to solve; every cell carries an approximate number and the margins are made consistent by construction. The trade is that a rounded value bounds the truth, so the base has to be large enough that the bound is not itself disclosive.

Cells and records hidden by primary and complementary suppression. what the repair costsprimary only960 cells · 1,992 recordsplus complementary964 cells · 2,016 records01000 cellsFour extra cells to close four exact recoveries spatialworkflow.io
On a wide table the repair is nearly free, which is why skipping it is indefensible.

Grand totals, percentages and map margins

  • The grand total is a margin. So is a number quoted in a press release.
  • Percentages publish the denominator. A rate and a count give the base.
  • Zeros are not holes. A published zero is a known value and helps the solver.
  • Store the suppression mask. Next release must reuse it, not recompute it.
  • Maps have margins too. A legend maximum publishes the largest cell.
  • Subtotals multiply the equations. Every subtotal is another constraint.
  • Check the published CSV, not the dashboard. Downloads usually carry more than the map does.
  • Document the rule. Users who know a blank means "<5" will not read a blank as zero.

FAQ

Why is suppressing small cells not enough?

Because a blanked cell in a row with a published total is a subtraction. Four rows in the test table had exactly one hole and were solved exactly.

What is complementary suppression?

Blanking additional cells so no suppressed value can be derived from a published margin. In practice: never leave exactly one hole in a row or column.

How much does complementary suppression cost?

On a wide table, very little. Repairing the London table cost three extra cells and eighteen extra hidden records.

What if a row has only one non-empty cell?

There is nothing to pair it with. Suppress the row total, or merge the row into a neighbouring area.

Do I need a solver, or is the heuristic enough?

The heuristic covers most open-data cases. If the table has subtotals, several margins, or a determined adversary, compute each suppressed cell's feasible interval with a linear programme.

How do time series leak suppressed cells?

A cell suppressed in one period and published in another, with a published total across periods, is recoverable by subtraction. Store the suppression mask and reuse it.