🔧 Fix Building permits & new housing Data cleaning Notebook By

Fixing Census Permit Files With Two Header Rows

On this page

Problem statement

The Census Bureau's building permit files look like CSV and are not quite. The first header row names the size classes, the second names the columns under them, the third line is blank, and the data rows have one more field than the header. pd.read_csv with its defaults reads the first row as the header and the second as data, turns every count into text, quietly moves the survey date into the index, and drops the leading zeros of the state and county codes. Nothing raises an error. This page shows each of those on the 2025 county file, the fix, and two further traps in older files and long names.

Quick answer

Skip the first three lines, supply the 30 column names, and read the codes as strings:

d = pd.read_csv(Z.open("co2025a.txt"), skiprows=3, header=None, names=COLS, skipinitialspace=True,
                dtype={"survey": str, "state": str, "county": str, "name": str})
print(d.shape, d.est1_units.dtype, d[["state", "county", "name"]].iloc[0].tolist())
(3029, 30) int64 ['01', '001', 'Autauga County                ']

3,029 rows, 30 columns, integer counts, and the codes 01 and 001 intact. The setup below wraps this in permits(), which also strips the 30-character padding from names.

The first four lines of the Census Bureau's co2025a.txt in monospace: a header row of size classes, a header row of column names, a blank line, and the first data row for Autauga County, each labelled. The first four lines of the Census Bureau's co2025a.txt in monospace: a header row of size classes, a header row of column names, a blank line, and the first data row for Autauga County, each labelled.
The first four lines of a county permit file: two header rows and a blank line.

What a plain read_csv does

1. The default read

naive = pd.read_csv(Z.open("co2025a.txt"))
print(naive.shape)
print(list(naive.columns[:10]))
print(naive.iloc[:2, :8].to_string())
(3030, 29)
['Survey', 'FIPS', 'FIPS.1', 'Region', 'Division', 'County', 'Unnamed: 6', '1-unit', 'Unnamed: 8', 'Unnamed: 9']
     Survey    FIPS FIPS.1 Region                        Division County Unnamed: 6    1-unit
Date  State  County   Code   Code                            Name  Bldgs      Units     Value
2025     01     001      3      6  Autauga County                    200        200  82131895

The frame has 3,030 rows and 29 columns: one row too many (the second header line) and one column too few. The header has 29 fields and the data rows 30, so pandas uses the first field of each row, the survey date, as the index: Date and 2025 sit where row numbers should be, and every column label is shifted one place from its data.

2. Every count becomes text

print(naive.dtypes.value_counts().to_string())
print(f"'1-unit' column summed: {str(naive['1-unit'].sum())[:40]}...")
str    29
'1-unit' column summed: Value82131895848168534402180015172153347...

All 29 columns are text, because the second header line put words like Units into every column. Summing the 1-unit column concatenates strings instead of adding: the result starts Value82131895....

Two horizontal bars of column types: a plain read_csv gives 29 text columns and no numeric ones, while skipping three rows and naming the columns gives 4 text and 26 numeric. Two horizontal bars of column types: a plain read_csv gives 29 text columns and no numeric ones, while skipping three rows and naming the columns gives 4 text and 26 numeric.
Column types read two ways: a plain read_csv leaves every count as text.

3. Codes lose their zeros

plain = pd.read_csv(Z.open("co2025a.txt"), skiprows=3, header=None)
print(plain.iloc[:2, :3].to_string(), "\n", plain[1].dtype)
      0  1  2
0  2025  1  1
1  2025  1  3 
 int64

With header=None the counts become numbers, but so do the codes: Alabama's 01 becomes 1 and Autauga's 001 becomes 1, so a five-digit GEOID can no longer be built and the join to boundaries fails.

Older files, names and odd rows

The data used on this page

The US Census Bureau's Building Permits Survey county files (annual 1990–2025 and year-to-date 2025–2026), its Northeast place files for 2020–2025, the Vintage 2025 county population estimates and the 2024 cartographic boundary counties, all public domain. They were downloaded on 16 October 2026 and saved on this site in one archive, because Census revises recent months as late reports arrive.

from pathlib import Path
import zipfile

import geopandas as gpd
import numpy as np
import pandas as pd
import requests

Path("data").mkdir(exist_ok=True)
# US Census Bureau Building Permits Survey: annual county files co1990a.txt-co2025a.txt, year-to-date county files
# co2501y.txt-co2512y.txt and co2601y.txt-co2608y.txt, Northeast place files ne2020a.txt-ne2025a.txt; the Vintage 2025
# county population estimates; 2024 cartographic boundary counties. Downloaded 16 October 2026, saved here unchanged.
# Live: https://www2.census.gov/econ/bps/County/co2025a.txt (and the same folder for the others),
# https://www2.census.gov/econ/bps/Place/Northeast%20Region/ne2025a.txt,
# https://www2.census.gov/programs-surveys/popest/datasets/2020-2025/counties/totals/co-est2025-alldata.csv,
# https://www2.census.gov/geo/tiger/GENZ2024/shp/cb_2024_us_county_20m.zip
ARCHIVE = "census-building-permits-2026-10-16.zip"
if not Path(f"data/{ARCHIVE}").exists():
    Path(f"data/{ARCHIVE}").write_bytes(requests.get(f"https://data.spatialworkflow.io/{ARCHIVE}", timeout=600).content)
Z = zipfile.ZipFile(f"data/{ARCHIVE}")

SIZES = ["1", "2", "34", "5"]  # buildings with 1 unit, 2 units, 3-4 units, 5 or more units
COLS = (["survey", "state", "county", "region", "division", "name"]
        + [f"{k}{s}_{m}" for k in ("est", "rep") for s in SIZES for m in ("bldgs", "units", "value")])


def permits(name):
    """One county file: two header rows and a blank line, then a row per county. est = estimate with
    imputation for places that did not report, rep = reported only."""
    d = pd.read_csv(Z.open(name), skiprows=3, header=None, names=COLS, skipinitialspace=True,
                    dtype={"survey": str, "state": str, "county": str, "name": str})
    d["survey"], d["name"] = d.survey.str.strip(), d.name.str.strip()
    d["state"], d["county"] = d.state.str.strip().str.zfill(2), d.county.str.strip().str.zfill(3)
    return d.assign(GEOID=d.state + d.county,
                    units=d[[f"est{s}_units" for s in SIZES]].sum(axis=1),
                    reported=d[[f"rep{s}_units" for s in SIZES]].sum(axis=1),
                    value=d[[f"est{s}_value" for s in SIZES]].sum(axis=1))


pop = pd.read_csv(Z.open("co-est2025-alldata.csv"), encoding="latin-1", dtype={"STATE": str, "COUNTY": str})
pop = pop[pop.SUMLEV == 50].assign(GEOID=lambda p: p.STATE + p.COUNTY)
Path("data/cb_2024_us_county_20m.zip").write_bytes(Z.read("cb_2024_us_county_20m.zip"))
counties = gpd.read_file("data/cb_2024_us_county_20m.zip")
counties = counties[~counties.STUSPS.isin(["AK", "HI", "PR"])].to_crs(5070)
y25 = permits("co2025a.txt")
print(f"2025 county file: {len(y25):,} rows, {y25.units.sum():,} units permitted; {len(pop):,} counties in the estimates")
2025 county file: 3,029 rows, 1,431,616 units permitted; 3,144 counties in the estimates

Example 1: the 1990s files

for line in Z.open("co1990a.txt").read().decode().splitlines()[:4]:
    print(line[:96])
old = permits("co1990a.txt")
print(old[["survey", "GEOID", "name", "units"]].head(2).to_string(index=False), "\n", old.units.dtype)
Survey , FIPS , FIPS , Region , Division , County ,   , 1-unit ,   ,   , 2-unit ,   ,   , 34unit
Date , State , County , Code , Code , Name , Bldgs  , Units , Value  , Bldgs  , Units , Value  ,

9099  ,     01  , 001  , 3  , 6  , Autauga County  , 87  , 87  , 4491772  , 0  , 0  , 0  , 0  , 
survey GEOID           name  units
  9099 01001 Autauga County     87
  9099 01003 Baldwin County    328 
 int64

The 1990s files have spaces around every comma and a survey code of 9099 for 1990. skipinitialspace=True and stripping the text columns handle the spaces, and the counts still come out as integers; take the year from the file name.

Example 2: names cut at 30 characters

ct = permits("co2025a.txt").query("state == '09'")
full = pop[pop.STATE == "09"].set_index("GEOID").CTYNAME
print(pd.DataFrame({"file": ct.set_index("GEOID").name, "estimates": full}).dropna().head(4).to_string())
print(f"names that would not match by text: {(ct.set_index('GEOID').name != full.reindex(ct.GEOID)).sum()} of {len(ct)}")
                                 file                                       estimates
GEOID                                                                                
09110         Capitol Planning Region                         Capitol Planning Region
09120  Greater Bridgeport Planning Re              Greater Bridgeport Planning Region
09130  Lower Connecticut River Valley  Lower Connecticut River Valley Planning Region
09140  Naugatuck Valley Planning Regi                Naugatuck Valley Planning Region
names that would not match by text: 8 of 9
Two columns of Connecticut planning region names, as stored in the permit file cut at 30 characters and in full from the population estimates, with the lost endings shown in red. Two columns of Connecticut planning region names, as stored in the permit file cut at 30 characters and in full from the population estimates, with the lost endings shown in red.
Region names cut at 30 characters: 8 of Connecticut's 9 do not match by text.

Names are cut at 30 characters: "Greater Bridgeport Planning Re" for Greater Bridgeport Planning Region. 8 of Connecticut's 9 region names in the 2025 file do not match the population estimates' names. Join on GEOID, never on the name.