⚙️ How-To Census & demographics Data sources & downloads Census & Demographics · step 4 of 20 Updated

How to Download Census Tables from an API in Python

Problem statement

Most tutorials for US census data start with one request to the Census Data API:

import requests

r = requests.get("https://api.census.gov/data/2023/acs/acs5",
                 params={"get": "NAME,B01003_001E", "for": "county:*", "in": "state:10"})
r.raise_for_status()
rows = r.json()

Run without an API key on 11 September 2026, that code did not return data. The API answered with a 302 redirect to missing_key.html. requests followed it and received an HTML page titled "Missing Key" with status 200, so raise_for_status() passed, and the failure only appeared one line later:

JSONDecodeError: Expecting value: line 1 column 1 (char 0)

The metadata endpoints still answered without a key — variable definitions, table groups and the list of supported geographies all returned JSON. And the Census Bureau publishes the same tables as bulk files that need no key at all: the whole country's tract populations downloaded and parsed in 1.7 s.

This guide covers both routes: the API done properly, with a key and with failures that name themselves, and the bulk summary files for when you want every geography at once.

Quick answer

Get a free key from the Census Bureau's key signup page, put it in an environment variable, and make the missing-key case raise a clear error:

import os
import requests
import pandas as pd

params = {"get": "NAME,B01003_001E,B01003_001M", "for": "tract:*", "in": "state:10",
          "key": os.environ["CENSUS_API_KEY"]}
r = requests.get("https://api.census.gov/data/2023/acs/acs5", params=params,
                 timeout=60, allow_redirects=False)
if r.is_redirect:
    raise RuntimeError(f"Census API redirected to {r.headers['Location']}")
r.raise_for_status()
rows = r.json()
frame = pd.DataFrame(rows[1:], columns=rows[0])       # every value arrives as text

Without a key, or when you need a table for every tract in the country, read the table-based summary file instead — no key, one request:

url = ("https://www2.census.gov/programs-surveys/acs/summary_file/2023/"
       "table-based-SF/data/5YRData/acsdt5y2023-b01003.dat")
table = pd.read_csv(url, sep="|", dtype=str)
tracts = table[table.GEO_ID.str.startswith("1400000US")]
print(len(tracts))                                     # 85381
Flow of a keyless Census Data API request: a 302 redirect to missing_key.html, a 200 HTML page, and a JSONDecodeError from .json().
The status code that should have failed the request is on the redirect, and requests follows redirects by default.

Step-by-step solution

1. Store the key outside the code

The key is free and arrives by email. Keep it in an environment variable or a secrets manager, never in a notebook or a repository:

export CENSUS_API_KEY="your-key-here"

Every data request needs key= in its parameters; a request without one gets the redirect. Activate the key from the link in the confirmation email before using it.

2. Discover variables without spending requests

Variable names are not guessable, and the metadata endpoints need no key. A table's group listing returns every variable in it:

group = requests.get("https://api.census.gov/data/2023/acs/acs5/groups/B17001.json",
                     timeout=60).json()["variables"]
estimates = sorted(v for v in group if v.endswith("E") and v != "NAME")
margins = sorted(v for v in group if v.endswith("M"))
print(len(group), len(estimates), len(margins))
238 59 59

B17001, poverty status by sex and age, has 59 estimates, 59 margins of error and 118 annotation variables, plus GEO_ID and NAME. Filter the names carefully: a suffix test for E also matches NAME, which is how a first attempt counted 60 estimates. Each variable also has its own record, with a readable label:

B17001_002E: Estimate!!Total:!!Income in the past 12 months below poverty level: | int

3. Build the geography from what the API requires

for names the level you want and in names its parents. The API's geography.json lists the 65 supported levels and what each requires:

county        requires state                      wildcard allowed for state
tract         requires state, county              wildcard allowed for county
block group   requires state, county, tract       wildcard allowed for county, tract
zip code tabulation area   requires nothing

So every tract in Delaware is for=tract:*&in=state:10, and every tract in the country is 52 calls — one each for the 50 states, the District of Columbia and Puerto Rico. That is the point at which the bulk file becomes the better route.

4. Turn the response into typed columns

The response is a JSON array of arrays, with a header row first. Every value is a string, including the numbers, and the geography comes back as separate state, county and tract columns:

frame["GEOID"] = frame["state"] + frame["county"] + frame["tract"]
for col in ("B01003_001E", "B01003_001M"):
    frame[col] = pd.to_numeric(frame[col])

Concatenate the geography columns as text, which keeps their leading zeros, and convert only the value columns to numbers. Then replace the annotation sentinels — values such as −666666666 — before any arithmetic.

5. Split long variable lists across requests

The API accepts at most 50 variables per request, and NAME counts. B17001's 59 estimates and 59 margins need three requests at 48 variables each. Merge the pieces on the geography columns, never by row position.

6. Use the bulk summary file for whole-country tables

The table-based summary file holds one pipe-delimited file per table, with every geography level in it, no key required:

B01003, tracts, over HTTP              85,381 rows in 1.7 s
B17001, four columns, tracts           85,381 rows in 3.7 s   (file 118.4 MB)
B19013, counties                        3,222 rows in 1.4 s

The same file read locally held 616,634 rows across 65 summary-level prefixes; reading B17001 with only the five columns needed took 0.91 s and 21 MB of memory. The naming differs from the API: the summary file calls the estimate B01003_E001 and the margin B01003_M001, where the API says B01003_001E and B01003_001M. Names for each GEO_ID are in a separate geography file, Geos20235YR.txt, with 616,701 rows.

7. Expect the sentinels in either route

Both routes carry the ACS annotation values. In the tract-level median household income table B19013, 1,547 estimates were −666666666, and the margins held 1,547 values of −222222222 and 492 of −333333333. A mean over that column without cleaning it is meaningless.

8. Outside the US

The same pattern works elsewhere, and neither of these needed a key when measured:

  • ONS census data for England and Wales is served by Nomis. A CSV request for LSOA usual residents returned in 2.1 s, keyed by codes such as E01011954.
  • Eurostat serves JSON-stat. A request for the 2023 population of DE300 (Berlin) and FR101 (Paris) returned 3,632,853 and 2,092,813 in 1.0 s.
Table of Census Data API geography levels with the parent geographies each requires in the in parameter and where wildcards are allowed.
Tracts need a state; block groups need a state and allow wildcards below it.

Code examples

Example 1 — an API call that fails with a reason 

import os

import pandas as pd
import requests

API = "https://api.census.gov/data"


class CensusKeyError(RuntimeError):
    pass


def census_get(dataset, variables, for_, in_=None, key=None, timeout=60):
    """One Census Data API call. Returns a DataFrame of strings."""
    key = key or os.environ.get("CENSUS_API_KEY")
    params = {"get": ",".join(variables), "for": for_}
    if in_:
        params["in"] = in_
    if key:
        params["key"] = key
    r = requests.get(f"{API}/{dataset}", params=params, timeout=timeout,
                     allow_redirects=False)
    if r.is_redirect and "key" in r.headers.get("Location", ""):
        raise CensusKeyError(f"{r.status_code} -> {r.headers['Location']}: "
                             "set CENSUS_API_KEY (free, from api.census.gov/data/key_signup.html)")
    r.raise_for_status()
    rows = r.json()
    return pd.DataFrame(rows[1:], columns=rows[0])

Without a key:

CensusKeyError: 302 -> https://api.census.gov/data/missing_key.html: set CENSUS_API_KEY (free, from api.census.gov/data/key_signup.html)

The error arrived in 0.5 s. allow_redirects=False is the important argument: with the default, the redirect is followed and the only symptom is an HTML body that is not JSON.

Example 2 — a whole table in batches under the variable limit 

def chunks(items, size):
    for i in range(0, len(items), size):
        yield items[i:i + size]


def census_group(dataset, group, for_, in_=None, per_call=48):
    """Every estimate and margin in a table group, merged on the geography columns."""
    meta = requests.get(f"{API}/{dataset}/groups/{group}.json", timeout=60).json()["variables"]
    wanted = sorted(v for v in meta if v.startswith(group) and v[-1] in "EM")
    geo_cols = None
    merged = None
    for batch in chunks(wanted, per_call):
        part = census_get(dataset, batch, for_, in_)
        geo_cols = [c for c in part.columns if c not in batch]
        part[batch] = part[batch].apply(pd.to_numeric)
        merged = part if merged is None else merged.merge(part, on=geo_cols, validate="one_to_one")
    print(f"{group}: {len(wanted)} variables in {-(-len(wanted) // per_call)} calls, {len(merged):,} rows")
    return merged

For B17001 the variable list has 118 entries — the startswith(group) test excludes NAME and GEO_ID, and the E/M test excludes the annotations — so the table arrives in three calls. The merge uses the geography columns and validate, so a batch that returned a different set of rows raises instead of misaligning values.

Example 3 — the keyless bulk route 

def acs_summary_table(table, level_prefix="1400000US", year=2023, columns=None):
    """Whole-country ACS 5-year table from the bulk summary file; no key needed."""
    url = (f"https://www2.census.gov/programs-surveys/acs/summary_file/{year}/"
           f"table-based-SF/data/5YRData/acsdt5y{year}-{table.lower()}.dat")
    usecols = None if columns is None else ["GEO_ID"] + columns
    frame = pd.read_csv(url, sep="|", dtype=str, usecols=usecols)
    frame = frame[frame.GEO_ID.str.startswith(level_prefix)].copy()
    frame.insert(0, "GEOID", frame.GEO_ID.str[len(level_prefix):])
    value_cols = [c for c in frame.columns if c not in ("GEO_ID", "GEOID")]
    frame[value_cols] = frame[value_cols].apply(pd.to_numeric)
    return frame.drop(columns="GEO_ID")
B01003 tracts over HTTP: 85,381 rows in 1.7 s
      GEOID  B01003_E001  B01003_M001
01001020100         1840          358
01001020200         2017          316

dtype=str keeps the identifiers intact before the value columns are converted. The same URL pattern answered for the 2022 and 2024 releases. Download the file once and read it locally when you use it repeatedly — B17001 alone is 118.4 MB.

Explanation

Why the failure hides in a 200

HTTP status codes describe the last response, and requests follows redirects unless told not to. The 302 that meant "no key" is in r.history; the response you hold is a successful delivery of an HTML page. So every check that looks at r.status_code or calls raise_for_status() passes, and the code fails on the JSON parse, with an error that mentions neither census data nor keys.

Turning redirects off for data calls makes the real response visible. It costs nothing, because a data endpoint has no legitimate reason to redirect.

Why metadata is free and data is not

The metadata endpoints — variables, groups, geography.json and the dataset catalogue — describe what exists and are cheap to serve. Data calls are what a key meters. That split is useful: you can discover variables, check geography requirements and validate a request plan before spending any authenticated calls.

Why every value is a string

The API returns one JSON format for every dataset, including text fields such as NAME and codes with leading zeros. Returning everything as strings keeps codes such as county 001 intact; it also means a numeric column is text until you convert it. Convert values explicitly, and build GEOID by concatenating the text geography columns.

When the bulk file is the better tool

An API call is right for a few variables in a few places. A national table at tract level would take one call per state for each batch of 48 variables, against one download that already contains every summary level. The bulk file also removes the key and rate limits from a pipeline entirely, at the price of downloading whole tables.

Decision diagram choosing between the Census Data API with a key and the bulk table-based summary file by scope of the request.
The API is for selective requests; the summary file is for whole tables at every level.

Edge cases or notes

  • An unactivated key behaves like no key. The confirmation email contains an activation link; until it is followed, requests redirect.
  • NAME ends in E. Filter variable lists by table prefix, not only by suffix.
  • Annotation variables end in EA and MA. They explain a sentinel value; request them only when you need the reason.
  • The summary file and the API name variables differently: B01003_E001 against B01003_001E.
  • Filter summary levels with the full prefix. 1400000US gives 85,381 tracts; shorter prefixes include partial geographies.
  • Vintages are directories. The catalogue listed 5-year ACS vintages through 2024; change the year in the path, and match the boundary file year.
  • Timeouts happen. One metadata request timed out after 60 s during testing; set a timeout and retry idempotent GETs.
  • Nomis and Eurostat use their own geography codes — E01… LSOA codes and NUTS codes such as DE300 — which do not need zero padding.

FAQ

Do I need an API key for the Census Data API?

For data requests, yes. Measured on 11 September 2026, a keyless request was redirected to missing_key.html. Metadata endpoints such as variables and geography.json still worked without one.

Why does my Census API request return JSONDecodeError?

The API probably redirected to its missing-key page, and requests followed the redirect to an HTML page with status 200. Pass allow_redirects=False to see the redirect, and add a key.

How many variables can I request at once?

Up to 50 per request, including NAME. Split larger tables into batches and merge the results on the geography columns.

How do I get data for every census tract in the US?

Either one API call per state, since tracts require a state, or one download of the table-based summary file, which read all 85,381 tracts of B01003 in 1.7 seconds with no key.

Why are all the numbers strings?

The API returns every value as text so that codes keep their leading zeros. Convert the value columns with pd.to_numeric and concatenate the geography columns into a GEOID as text.

Is there an equivalent for UK or EU census data?

Nomis serves ONS census tables as CSV, and Eurostat serves regional statistics as JSON-stat. Both answered without a key when tested.