🧭 Concept Census & demographics Spatial statistics Census & Demographics · step 10 of 20 Updated

Small Numbers Explained: Why Rates in Small Areas Jump Around

Problem statement

Sort census tracts by any rate — poverty, unemployment, a disease — and the top of the list fills with small places. So does the bottom. It looks like a finding: small communities are more extreme. Mostly, it is arithmetic.

A rate computed from a small denominator can only take coarse values, and a few people more or less moves it a long way. Measured on the 84,341 US tracts with a poverty universe in the 2023 ACS 5-year estimates:

  • All 33 tracts with a poverty rate of exactly 100% are in the smallest tenth of tracts by population, and 65.4% of the 280 tracts at exactly 0% are too.
  • 44.0% of the highest 1% of rates come from that smallest tenth, four times the 10% you would expect if size did not matter.
  • Rates in the smallest tenth spread nearly twice as widely as in the largest: a standard deviation of 16.8 percentage points against 8.5.

Rates in small areas are not wrong. They are noisy, and on a map the noise looks exactly like pattern.

Quick answer

Before mapping or ranking a rate, look at how its spread depends on the denominator:

import pandas as pd

tracts["rate"] = tracts["below"] / tracts["universe"]
tracts["size_decile"] = pd.qcut(tracts["universe"], 10, labels=False) + 1
print(tracts.groupby("size_decile")["rate"].agg(["std", "min", "max"]))

If the smallest decile is much more spread out, or holds most of the extremes, the map will show where small areas are rather than where the rate is high. The standard responses, in rough order of effort:

  • Flag unreliable rates with their margin of error or a minimum denominator.
  • Aggregate small areas into larger ones until the denominators are adequate.
  • Smooth rates towards a regional average by how much evidence each area has — Empirical Bayes.
Funnel plot of the 95% range a tract poverty rate would take by chance around the pooled rate of 12.73%, narrowing as the denominator grows.
With 100 people, chance alone moves a rate 6.5 points either side of the average; with 4,000, only 1 point.

Step-by-step solution

1. Recognise a small-number rate

A rate's possible values are set by its denominator. With 5 people, the only possible poverty rates are 0%, 20%, 40%, 60%, 80% and 100%. With 50 there are 51 possible values, 2 points apart. With 4,000 the steps are too small to see.

The ACS tract data contain the whole range. The smallest tenth of tracts had poverty universes from 1 to 1,899 people, and 179 tracts had fewer than 50 — and among those 179, 70.4% had a rate of exactly 0% or exactly 100%.

2. Separate noise from real variation

Some of the spread in small areas is real: small tracts include institutions, rural areas and new developments that genuinely differ. To see how much is chance, simulate a world in which every tract has the national rate and only the denominators differ:

tract poverty rate, pooled 12.73%
size decile   denominator      observed SD   SD from chance alone
     1          1 – 1,899        16.8 pts           1.9 pts
     5      3,287 – 3,690        10.6 pts           0.6 pts
    10      6,086 – 34,767        8.5 pts           0.4 pts

Chance alone produced nearly five times the spread in the smallest decile as in the largest. The observed spread is much larger than chance everywhere — tracts really do differ — but the extra spread in small tracts is where the chance component concentrates.

3. Check where the extremes come from

The highest and lowest values on a map draw the eye first, and they are exactly where small denominators dominate:

highest 1% of poverty rates (843 tracts)
  share from the smallest decile, observed               44.0%
  share from the smallest decile, if only chance         56.5%
  share if denominator size made no difference           10.0%

The simulation shows that in a world with no real differences at all, more than half the top 1% would still be small tracts. The observed 44.0% is lower only because real high-poverty tracts of ordinary size compete for the top places.

4. Remember that survey estimates add more noise

The simulation counts only the noise of a binomial draw on a known denominator. ACS rates also carry sampling error from the survey itself, and the published margins show it. The median margin of error of a tract poverty rate fell steadily with size, from 7.6 points in the smallest decile to 4.4 in the largest. Registry data — deaths, cancer cases, crimes — have no survey margin, but the same small-denominator noise.

5. Watch it get worse at finer levels

Block groups are a third the size of tracts. For the renter share of households, the smallest tenth of block groups had 256 or fewer households, 11,491 block groups had a renter share of exactly 0%, and 5,333 had exactly 100%. The median margin of error of the renter share in the smallest block groups was 17.4 points.

6. Decide how to respond

  • For a table or report: publish the denominator and the margin of error next to every rate, and suppress or flag rates below a minimum denominator.
  • For a map: hatch or fade unreliable areas, or aggregate them first.
  • For ranking or hotspot detection: smooth the rates, because a ranking takes the extremes at face value.
Bar chart of the share of the highest 1% of tract poverty rates that come from the smallest tenth of tracts: observed, under chance alone, and if size made no difference.
The top of a ranked list is where small denominators are most over-represented.

Code examples

Example 1 — spread by denominator size, against chance 

import numpy as np
import pandas as pd


def spread_by_size(frame, numerator, denominator, bins=10, seed=42):
    """Observed spread of a rate by denominator decile, next to the spread from chance alone."""
    df = frame[frame[denominator] > 0].copy()
    n, k = df[denominator].to_numpy(), df[numerator].to_numpy()
    df["rate"] = k / n
    pooled = k.sum() / n.sum()
    df["decile"] = pd.qcut(df[denominator], bins, labels=False, duplicates="drop")
    df["chance"] = np.random.default_rng(seed).binomial(n, pooled) / n
    rows = []
    for d, g in df.groupby("decile"):
        rows.append((int(d) + 1, int(g[denominator].min()), int(g[denominator].max()),
                     g.rate.std() * 100, (g.rate.quantile(.9) - g.rate.quantile(.1)) * 100,
                     (g.rate == 0).mean() * 100, (g.rate == 1).mean() * 100, g.chance.std() * 100))
    out = pd.DataFrame(rows, columns=["decile", "n_min", "n_max", "sd_pp", "p10_p90_pp",
                                      "zero_%", "one_%", "null_sd_pp"])
    print(f"{len(df):,} units with denominator > 0; pooled rate {pooled * 100:.2f}%")
    print(out.round(1).to_string(index=False))
    return df
84,341 units with denominator > 0; pooled rate 12.73%
 decile  n_min  n_max  sd_pp  p10_p90_pp  zero_%  one_%  null_sd_pp
      1      1   1899   16.8        38.8     2.2    0.4         1.9
      2   1900   2439   12.7        30.1     0.2    0.0         0.7
      3   2440   2878   11.6        27.6     0.1    0.0         0.6
      4   2879   3286   11.3        26.1     0.2    0.0         0.6
      5   3287   3690   10.6        24.3     0.1    0.0         0.6
      6   3691   4109   10.1        23.3     0.1    0.0         0.5
      7   4110   4590    9.9        22.8     0.2    0.0         0.5
      8   4591   5190    9.7        21.6     0.1    0.0         0.5
      9   5191   6085    9.0        20.3     0.1    0.0         0.4
     10   6086  34767    8.5        18.3     0.2    0.0         0.4

The null_sd_pp column is the spread you would see if every tract had the pooled rate. The ratio between the first and last rows of that column is the part of the pattern that size alone creates.

Example 2 — where the extremes come from 

def extremes_by_size(df, share=0.01):
    """How much of the top of a ranking comes from the smallest decile, observed and by chance."""
    m = int(len(df) * share)
    observed = (df.nlargest(m, "rate")["decile"] == 0).mean()
    chance = (df.nlargest(m, "chance")["decile"] == 0).mean()
    print(f"highest {share:.0%} of rates ({m}): smallest decile holds {observed:.1%} observed, "
          f"{chance:.1%} under chance alone")
highest 1% of rates (843): smallest decile holds 44.0% observed, 56.5% under chance alone
highest 5% of rates (4217): smallest decile holds 28.3% observed, 28.8% under chance alone

Pass the frame returned by Example 1. At the top 5%, the observed share and the chance share are nearly equal: a ranking that deep is shaped as much by denominators as by the rate.

Example 3 — the tiny denominators 

def tiny_denominators(df, denominator, threshold=50):
    """Rates from very small denominators: how many, how spread, how many at 0 or 100%."""
    tiny = df[df[denominator] < threshold]
    at_limit = ((tiny.rate == 0) | (tiny.rate == 1)).mean()
    print(f"units with denominator < {threshold}: {len(tiny):,}; their rate SD {tiny.rate.std() * 100:.1f} pp; "
          f"share at 0 or 100%: {at_limit:.1%}")
    return tiny
units with denominator < 50: 179; their rate SD 38.7 pp; share at 0 or 100%: 70.4%

A standard deviation of 38.7 points on a scale of 0 to 100 means these rates are close to uninformative. Suppress them, or merge the areas with neighbours, before any map is drawn.

Explanation

Why small denominators produce extreme rates

A rate is a count divided by the number of people who could have been counted. If each person independently has the same chance of being counted, the count follows a binomial distribution, and the standard error of the rate is the square root of p(1 − p)/n. Quadruple the denominator and the chance spread halves.

At the pooled poverty rate of 12.73%, that puts the 95% range of a rate from chance alone at about ±6.5 points for 100 people, ±2.1 for 1,000 and ±1.0 for 4,000. A tract of 100 people can land at 20% poverty with nothing unusual about it at all.

Why the map draws attention to the noise

Choropleth colour classes put the most extreme values in the darkest and lightest colours. Those values come disproportionately from small denominators, so the most prominent areas on the map are the least reliable. In rural regions, small-denominator areas are also large on the ground, so the noisiest rates cover the most paper.

Why real variation does not rescue the pattern

Observed spread was larger than chance in every size decile, so tract poverty rates do vary for real reasons. That does not tell you which extreme values are real. A tract at 100% might be a college dormitory block or three people in a sampled household; the rate alone cannot say. Methods that weigh the rate against its denominator — funnel plots, margins of error, Empirical Bayes smoothing — make that judgement explicit instead of leaving it to the colour scale.

Why this is the same problem as the margin of error

Both describe how much a rate would change with a different set of people. The margin of error measures the survey's sampling of households; the binomial spread measures the chance element in any small count. For survey data both apply, which is why the published rate margins shrink as tracts get larger.

Table comparing the smallest, middle and largest tenth of tracts by denominator range, observed spread, spread from chance and median margin of error of the poverty rate.
Every column shrinks with size, but the chance column shrinks fastest.

Edge cases or notes

  • Exactly 0% and 100% are small-number signatures. All 33 tracts at 100% poverty were in the smallest size decile.
  • Group quarters create tiny universes. Poverty status is not determined for people in institutions, so a large tract can have a poverty universe of a handful.
  • Real variation can be large too. Renter share varies so much for real reasons that its observed spread barely changes with size: 27.5 points in the smallest decile against 23.5 in the largest.
  • The chance model is a lower bound for survey data. ACS sampling error comes on top of it.
  • A minimum denominator is a blunt rule. It removes noise and also real small communities; state the threshold.
  • Aggregation trades detail for stability. Merged areas are more reliable and describe a larger, more mixed place.
  • Hotspot and other local statistics inherit the problem. Smooth or test rates before feeding them in.

FAQ

Why do small areas have the highest and lowest rates?

Because a small denominator lets a few people move the rate a long way. All 33 US tracts with a 100% poverty rate were in the smallest tenth by population, and chance alone would put more than half of the top 1% of rates there.

How small is too small for a rate?

It depends on the rate and the use, but the spread is easy to see. Tracts with a poverty universe under 50 had a rate standard deviation of 38.7 points, and 70.4% of them sat at exactly 0% or 100%.

Is the variation in small areas just noise?

Not all of it. Observed spread exceeded the chance spread in every size group, so real differences exist. The problem is that the extremes mix real and chance values, and the rate alone cannot separate them.

How do I fix unstable rates on a map?

Flag or suppress rates with small denominators, aggregate small areas, or smooth the rates with Empirical Bayes, which pulls each rate towards the regional average in proportion to how little evidence it rests on.

Does this apply to data that is not a survey?

Yes. Death rates, crime rates and disease rates from complete registers still follow the binomial arithmetic; only the survey's own sampling error is absent.

What is a funnel plot?

A chart of rate against denominator with chance limits drawn around the average. The limits narrow as the denominator grows, so areas outside the funnel are unusual for their size rather than merely small.