🧭 Concept Census & demographics Spatial statistics Census & Demographics · step 8 Notebook By · Updated

Margins of Error Explained: Why Survey Estimates Need Their Uncertainty

On this page

Problem statement

Every number in the American Community Survey comes with a second number. B17001_E002 is the estimated count of people below the poverty line; B17001_M002 is its margin of error. Most maps, dashboards and spreadsheets use the first column and drop the second.

At county level that is a forgivable shortcut. At tract level it is not. Measured on the 2023 ACS 5-year poverty table:

  • The median tract estimate of people in poverty has a coefficient of variation of 34.1%, against 9.9% for the median county.
  • Only 212 of 85,381 tract estimates count as highly reliable by the common 12% rule; 27,251 are low reliability.
  • Only 45.7% of tracts have a poverty rate statistically different from their county's — so for more than half the tracts on a map, the colour difference from the county average is not supported by the data.

A margin of error is not an optional extra column. It is the part of the estimate that says how much of the map to believe.

Quick answer

ACS margins of error are 90% margins. Convert them to a standard error and a coefficient of variation before deciding anything:

se = moe / 1.645                    # standard error
cv = se / estimate                  # coefficient of variation
low, high = estimate - moe, estimate + moe

A tract with 100 people below poverty and a margin of 68 has a standard error of 41.3, a CV of 41.3%, and a 90% interval from 32 to 168. That estimate cannot tell 50 people from 150.

Use the CV to flag reliability, and test differences with both margins before claiming that two places differ:

different = abs(est1 - est2) / ((moe1 / 1.645) ** 2 + (moe2 / 1.645) ** 2) ** 0.5 > 1.645
An ACS estimate of 100 with its 90% interval from 32 to 168 and its 95% interval from 19 to 181. estimate 100, published MOE 6805010015020095%: 19 to 181 (± 81)90% (published): 32 to 168SE = 68 / 1.645 = 41.3 · CV = 41.3% · low reliability spatialworkflow.io
The published margin is a 90% interval; a 95% interval is 1.19 times wider.

What a margin measures, and judging it by the CV

1. Know what the margin measures

The ACS is a survey of a sample of addresses, pooled over five years for the 5-year release. Each estimate would come out differently with a different sample, and the margin of error describes that sampling variation. At 90% confidence, the interval estimate ± MOE is expected to contain the value a complete count would give in nine samples out of ten.

The margin covers sampling error only. Non-response, coding mistakes and the gap between a five-year average and a single year are not in it.

2. Convert the margin to the scale you need

standard error          SE  = MOE / 1.645
coefficient of variation CV = SE / estimate
95% margin              MOE95 = MOE × 1.96 / 1.645 = MOE × 1.1915

Other agencies publish 95% intervals, so convert before comparing an ACS margin with one from elsewhere.

3. Judge reliability by the CV, not the margin

A margin of 68 is large for an estimate of 100 and trivial for 100,000. The CV removes the scale. A widely used convention treats a CV up to 12% as high reliability, 12–40% as medium and above 40% as low:

estimate: people below poverty    high      medium     low      zero estimate
tracts                              212     56,598   27,251          1,320
counties                          2,059      1,138       25              —

The median CV was 34.1% for tracts and 9.9% for counties. A third of tract estimates (32.4%) had a CV above 40%, and 5.9% had a margin at least as large as the estimate itself.

4. Expect reliability to depend on the variable and the level

Totals are more reliable than subgroups, and larger areas more than smaller ones:

median CV                         county    tract    block group
people below poverty                9.9%    34.1%    not published
median household income             4.1%    12.4%       20.0%
total population                      —      8.4%       18.6%

Median household income is published for 221,369 block groups, and a quarter of them had a CV above 30%. The poverty table is not published for block groups at all.

5. Recognise estimates that have no margin

County total population is missing from the county row above for a reason. Its margin is the sentinel −555555555 in 3,090 of 3,222 counties and in all 52 states. That code means the estimate is controlled to the Census Bureau's official population estimates, so a sampling margin does not apply. The 132 counties with a real margin are those that are not controlled this way, with populations from 43 to 164,632.

Treat −555555555 as "no sampling error to report", never as a number.

6. Do not treat a zero as certain

1,320 tracts have an estimate of zero people below poverty, and their margins are not zero — the median is 13. The survey found nobody, which is consistent with a small number who were not sampled. Map zeros as "fewer than about the margin", not as a firm absence.

7. Test differences before describing them

Two estimates differ at 90% confidence when their difference is larger than 1.645 standard errors of the difference:

z = abs(p1 - p2) / ((moe1 / 1.645) ** 2 + (moe2 / 1.645) ** 2) ** 0.5

Applied to every tract with at least 100 people in the poverty universe, 45.7% had a poverty rate significantly different from their county's, and for 53.3% the tract's own 90% interval contained the county rate. A choropleth colours every one of those tracts differently from the county average anyway.

8. Be careful with rankings

Ranks magnify uncertainty. In Cook County, Illinois, 1,328 tracts have a poverty universe of at least 100. The 90% interval of the median-ranked tract overlapped the intervals of 1,152 of them — 87%. A ranked list of tracts by poverty rate is mostly an ordering of noise in its middle.

Median coefficients of variation for ACS poverty counts, household income and total population by geographic level. median coefficient of variation, 2023 ACS 5-yearpoverty count, county9.9%poverty count, tract34.1%income, county4.1%income, tract12.4%income, block group20.0%population, block group18.6%040%Subgroups in small areas are the least reliable estimates spatialworkflow.io
The same survey is reliable for counties and unreliable for most tracts, because the sample behind each tract is small.

Reliability, rate margins and real differences

The data used on this page

Three tables from the 2023 ACS 5-year table-based summary file: B17001 poverty status (the universe and the count below poverty), B19013 median household income and B01003 total population. The setup keeps state, county, tract and block-group rows and labels each with its level; the county of a tract or block group is the first five digits of its GEOID.

import os
import urllib.request

import numpy as np
import pandas as pd

BASE = "https://www2.census.gov/programs-surveys/acs/summary_file/2023/table-based-SF/data/5YRData/"
LEVELS = {"0400000US": "state", "0500000US": "county", "1400000US": "tract", "1500000US": "block group"}


def acs(table, columns):
    path = f"acsdt5y2023-{table}.dat"
    if not os.path.exists(path):
        urllib.request.urlretrieve(BASE + path, path)
    t = pd.read_csv(path, sep="|", usecols=["GEO_ID", *columns], dtype={"GEO_ID": str})
    t["level"] = t.GEO_ID.str[:9].map(LEVELS)
    t = t.dropna(subset=["level"])
    t["geoid"] = t.GEO_ID.str[9:]
    return t


pov = acs("b17001", ["B17001_E001", "B17001_M001", "B17001_E002", "B17001_M002"])
inc = acs("b19013", ["B19013_E001", "B19013_M001"])
pop = acs("b01003", ["B01003_E001", "B01003_M001"])
print(pov.level.value_counts().to_dict(), inc.level.value_counts().to_dict())
{'tract': 85381, 'county': 3222, 'state': 52} {'block group': 242296, 'tract': 85381, 'county': 3222, 'state': 52}

Example 1 — reliability for a whole column

import numpy as np
import pandas as pd

Z90 = 1.645
SENTINELS = [-222222222, -333333333, -555555555, -666666666, -888888888, -999999999]


def reliability(estimate, moe, bands=(0.12, 0.40)):
    """Standard error, coefficient of variation and a reliability class for ACS estimates."""
    est = estimate.where(~estimate.isin(SENTINELS))
    m = moe.where(~moe.isin(SENTINELS))
    se = m / Z90
    cv = se / est.where(est > 0)
    label = pd.cut(cv, [0, bands[0], bands[1], np.inf], labels=["high", "medium", "low"], include_lowest=True)
    label = label.cat.add_categories(["no MOE", "zero estimate"])
    label[m.isna()] = "no MOE"
    label[(est == 0) & m.notna()] = "zero estimate"
    print(label.value_counts().reindex(["high", "medium", "low", "zero estimate", "no MOE"]).to_dict())
    return pd.DataFrame({"estimate": est, "moe": m, "se": se, "cv": cv, "reliability": label})


tr, co = pov[pov.level == "tract"], pov[pov.level == "county"]
print("tract people below poverty:")
tract_rel = reliability(tr.B17001_E002, tr.B17001_M002)
print("county people below poverty:")
county_rel = reliability(co.B17001_E002, co.B17001_M002)
print("county total population:")
pop_rel = reliability(pop[pop.level == "county"].B01003_E001, pop[pop.level == "county"].B01003_M001)
tract people below poverty:
{'high': 212, 'medium': 56598, 'low': 27251, 'zero estimate': 1320, 'no MOE': 0}
county people below poverty:
{'high': 2059, 'medium': 1138, 'low': 25, 'zero estimate': 0, 'no MOE': 0}
county total population:
{'high': 121, 'medium': 10, 'low': 1, 'zero estimate': 0, 'no MOE': 3090}

Sentinels become missing before any arithmetic, so a −555555555 margin cannot produce a negative CV. Keep the reliability column next to the estimate all the way to the map.

Example 2 — the margin of a rate

def proportion_moe(num, moe_num, den, moe_den):
    """MOE of num/den when num is a subset of den; falls back to the ratio formula."""
    p = num / den
    radicand = moe_num ** 2 - p ** 2 * moe_den ** 2
    ratio = moe_num ** 2 + p ** 2 * moe_den ** 2
    used_ratio = radicand < 0
    moe = np.sqrt(np.where(used_ratio, ratio, radicand)) / den
    print(f"{int(used_ratio.sum())} of {len(p):,} used the ratio formula")
    return p, pd.Series(moe, index=num.index)


t100 = tr[tr.B17001_E001 >= 100]                       # tracts with a poverty universe of 100+
p_t, moe_t = proportion_moe(t100.B17001_E002, t100.B17001_M002, t100.B17001_E001, t100.B17001_M001)
p_c, moe_c = proportion_moe(co.B17001_E002, co.B17001_M002, co.B17001_E001, co.B17001_M001)
print(f"median tract rate MOE pp {100 * moe_t.median():.1f} county {100 * moe_c.median():.1f}")
95 of 84,088 used the ratio formula
0 of 3,222 used the ratio formula
median tract rate MOE pp 5.6 county 2.3

This is the Census Bureau's approximation for a proportion whose numerator is part of its denominator. When the quantity under the square root is negative — 95 tracts here — the documented fallback is the ratio formula, which adds the terms instead. The median tract poverty rate carries a margin of 5.6 percentage points.

Example 3 — which differences are real

def differ(est1, moe1, est2, moe2, z=Z90):
    """Are two independent estimates different at the confidence level of their MOEs?"""
    stat = (est1 - est2).abs() / np.sqrt((moe1 / Z90) ** 2 + (moe2 / Z90) ** 2)
    return stat > z


county_of = t100.geoid.str[:5]
cp = pd.Series(p_c.values, index=co.geoid.values).reindex(county_of).values
cm = pd.Series(moe_c.values, index=co.geoid.values).reindex(county_of).values
d = differ(p_t, moe_t, pd.Series(cp, index=p_t.index), pd.Series(cm, index=p_t.index))
print(f"tracts different from their county: {d.mean():.1%} of {len(d):,}")
tracts different from their county: 45.7% of 84,088

The test assumes the two estimates are independent. A tract and the county containing it are not, because the tract's sample is part of the county's, so the test is conservative for this comparison. Pass z=1.96 to test at 95% confidence while keeping the published 90% margins as input.

Example 4 — the other figures on this page

cv_of = lambda est, moe: (moe / Z90) / est.where(est > 0)
print("median CV")
print(f"  people below poverty      county {cv_of(co.B17001_E002, co.B17001_M002).median():.1%}  "
      f"tract {tract_rel.cv.median():.1%}")
for lvl in ("county", "tract", "block group"):
    x = inc[(inc.level == lvl) & (inc.B19013_E001 > 0) & (inc.B19013_M001 > 0)]
    y = pop[(pop.level == lvl) & (pop.B01003_M001 > 0)]
    print(f"  {lvl:12} median income {cv_of(x.B19013_E001, x.B19013_M001).median():5.1%} "
          f"({len(x):,} published)   total population {cv_of(y.B01003_E001, y.B01003_M001).median():5.1%}")
bg_inc = inc[(inc.level == "block group") & (inc.B19013_E001 > 0) & (inc.B19013_M001 > 0)]
print(f"block groups with income CV over 30%: {(cv_of(bg_inc.B19013_E001, bg_inc.B19013_M001) > 0.3).mean():.1%}")

nz = tr[tr.B17001_E002 > 0]                             # non-zero estimates
print(f"\nnon-zero tract poverty estimates: CV above 40% {(tract_rel.cv.dropna() > 0.40).mean():.1%}; "
      f"margin at least the estimate {(nz.B17001_M002 >= nz.B17001_E002).mean():.1%}")
zero = tr[tr.B17001_E002 == 0]
print(f"zero-estimate tracts {len(zero):,}, median margin {zero.B17001_M002.median():.0f}")

cpop, spop = pop[pop.level == "county"], pop[pop.level == "state"]
real = cpop[cpop.B01003_M001 > 0]
print(f"\npopulation margin -555555555: {int((cpop.B01003_M001 == -555555555).sum()):,} of {len(cpop):,} counties, "
      f"{int((spop.B01003_M001 == -555555555).sum())} of {len(spop)} states; the {len(real)} with a real margin "
      f"range {real.B01003_E001.min():,} to {real.B01003_E001.max():,} people")

inside = (np.abs(p_t - cp) <= moe_t)
print(f"\ntracts whose own 90% interval contains the county rate: {inside.mean():.1%}")

cook = county_of == "17031"
lo, hi = (p_t - moe_t)[cook], (p_t + moe_t)[cook]
mid = p_t[cook].rank(method="first").sub(cook.sum() / 2).abs().idxmin()
overlap = ((lo <= hi[mid]) & (hi >= lo[mid])).sum() - 1
print(f"Cook County: {int(cook.sum()):,} tracts with a universe of 100+; the median-ranked tract's interval "
      f"overlaps {overlap:,} of them ({overlap / cook.sum():.0%})")
median CV
  people below poverty      county 9.9%  tract 34.1%
  county       median income  4.1% (3,220 published)   total population  6.1%
  tract        median income 12.4% (83,342 published)   total population  8.4%
  block group  median income 20.0% (221,369 published)   total population 18.6%
block groups with income CV over 30%: 25.3%

non-zero tract poverty estimates: CV above 40% 32.4%; margin at least the estimate 5.9%
zero-estimate tracts 1,320, median margin 13

population margin -555555555: 3,090 of 3,222 counties, 52 of 52 states; the 132 with a real margin range 43 to 164,632 people

tracts whose own 90% interval contains the county rate: 53.3%
Cook County: 1,328 tracts with a universe of 100+; the median-ranked tract's interval overlaps 1,152 of them (87%)

Why tract margins are so wide

Why tract margins are so wide

The precision of a survey estimate depends mostly on how many responses sit behind it, not on the population it describes. A county's estimate draws on thousands of sampled households; a tract of 4,000 people draws on a small fraction of that. Subgroups are thinner still: people below poverty in a tract are a subset of a subset. That is why the median CV rises from 9.9% for county poverty counts to 34.1% for tracts, and why block-group poverty is not published at all.

Why the CV is the right yardstick

A margin in absolute units cannot be compared across estimates of different sizes. Dividing the standard error by the estimate expresses the uncertainty as a share of the value, so a 12% threshold means the same thing for 100 people and for 100,000. It fails only near zero, where the CV grows without limit — which is why zero and near-zero estimates need their own treatment.

Why overlapping intervals are not a test

Two 90% intervals can overlap and the estimates still be significantly different, because the standard error of a difference is smaller than the sum of the two standard errors. Comparing intervals by eye is conservative; the z-test in Example 3 is the correct check, and it is one line.

Why controlled estimates have no margin

The ACS weights its responses so that county totals of population and housing match the Census Bureau's separate population estimates. Those totals are therefore fixed by the weighting rather than estimated from the sample, and the survey has no sampling error to report for them. The sentinel −555555555 says exactly that. Characteristics within those totals — poverty, income, age groups — still have sampling error and still carry margins.

Measured tests of tract poverty rates against their county and among Cook County tracts. tract poverty rates with their marginscomparisonresultdiffers from its county (z > 1.645)45.7% of tractstract interval contains county rate53.3% of tractsCook County: median tract overlaps1,152 of 1,328 (87%)median tract rate MOE5.6 points (county 2.3)tracts with a poverty universe of at least 100Ranking tracts by rate orders mostly noise in the middle spatialworkflow.io
More than half the tracts on a poverty map are indistinguishable from their county average at 90% confidence.

Confidence levels, coded margins and zero estimates

  • Margins are published at 90%. Multiply by 1.1915 for 95%; divide by 1.645 for a standard error.
  • Negative sentinel margins are codes, not values. −555555555 means controlled; others mean the margin could not be calculated or the estimate is suppressed.
  • Zero estimates have positive margins. The median for tracts with no one below poverty was 13.
  • Margins cannot be summed. Aggregating estimates needs the root-sum-of-squares approximation, which has its own biases.
  • The reliability bands are a convention. 12% and 40% are widely used, not official; state the bands you apply.
  • A rate's margin is not the count's margin divided by the denominator. Use the proportion formula, with the ratio fallback.
  • 5-year estimates are period averages. They describe the five years pooled, not the final year.
  • Other surveys publish other confidence levels. Convert to a common level before comparing.

FAQ

What confidence level do ACS margins of error use?

90%. Divide the margin by 1.645 to get a standard error, or multiply it by 1.1915 to get a 95% margin.

How do I know whether an ACS estimate is reliable?

Compute its coefficient of variation: the margin divided by 1.645, divided by the estimate. A common convention treats up to 12% as high reliability and above 40% as low; the median tract poverty count was 34.1%.

Why does my county population have a margin of −555555555?

The estimate is controlled to the Census Bureau's official population estimates, so there is no sampling error to report. That applied to 3,090 of 3,222 counties in the 2023 5-year release.

Can I say one tract has a higher poverty rate than another?

Only after a significance test using both margins. For 45.7% of tracts, the poverty rate was significantly different from the county rate; for the rest, the data cannot support the difference.

Why does an estimate of zero have a margin of error?

Because the survey samples households. Finding nobody in the sample is consistent with a few people in the population; the median margin for tracts with a zero poverty count was 13.

Should I use block groups for more detail?

Only for variables published at that level and with their margins checked. Median household income at block-group level had a median CV of 20.0%, and the poverty table is not published for block groups.