Crash Data Coverage Explained

Problem statement

Every crash dataset is defined by what it leaves out. FARS contains only crashes in which someone died within 30 days, on a public road. STATS19 contains only collisions in which someone was injured and the police were told. A city's crash records contain what its own police department reported, which can include minor damage-only crashes and exclude roads patrolled by another agency. Two datasets describing the same place and year can therefore disagree, and both be right about what they cover.

This page lays out what goes into three datasets, using Chicago as the test: the city's 2024 crash records, broken down by how they were reported and whether anyone was hurt, and a crash-by-crash match between FARS 2023's fatal crashes in Chicago and the fatal crashes in the city's own records for the same year.

Quick answer

Before comparing two crash datasets, match them record by record on date and location, and look at what fails to match:

M = FARS_CHI.copy()
M["matched"] = [((abs((CITY_FATAL.date - d).dt.days) <= 1) & (CITY_FATAL.distance(g) < 1640)).any()   # same or next day, within 500 m
                for d, g in zip(M.date, M.geometry)]
print(f"FARS 2023 fatal crashes in Chicago: {len(M)}; city records with a death in 2023: {len(CITY_FATAL[CITY_FATAL.date.dt.year == 2023])}")
print(f"FARS crashes matched in the city records: {M.matched.sum()} ({M.matched.mean():.0%})")
print(M.groupby("matched").FUNC_SYSNAME.value_counts().unstack(0).fillna(0).astype(int).sort_values(False, ascending=False).head(5).to_string())

FARS 2023 fatal crashes in Chicago: 159; city records with a death in 2023: 142
FARS crashes matched in the city records: 101 (64%)
matched                   False  True 
FUNC_SYSNAME                          
Interstate                   25      0
Major Collector               9     32
Minor Arterial                9     28
Other Principal Arterial      8     33
Local                         7      8

FARS has 159 fatal crashes in Chicago in 2023; the city's records have 142 crashes with a death and usable coordinates. 101 of FARS's crashes, 64%, have a city record within 500 metres and a day. All 25 FARS crashes on interstates are among the unmatched; on other roads most crashes match.

What city records hold, damage thresholds, who reports which roads

1. What the city records contain

Chicago's records include every crash its police reported, with or without injury. Two fields describe how the crash entered the data:

t = C.groupby(["report_type", "crash_type"], dropna=False).size().unstack()
print(t.to_string())
desk = C.report_type == "NOT ON SCENE (DESK REPORT)"
print(f"desk reports, made without an officer at the scene: {desk.mean():.1%}; of them with injury or tow: "
      f"{(C.crash_type[desk] == 'INJURY AND / OR TOW DUE TO CRASH').mean():.1%}")

crash_type                  INJURY AND / OR TOW DUE TO CRASH  NO INJURY / DRIVE AWAY
report_type                                                                         
NOT ON SCENE (DESK REPORT)                              4942                   52107
ON SCENE                                               25426                   24512
NaN                                                     2169                    2901
desk reports, made without an officer at the scene: 50.9%; of them with injury or tow: 8.7%

Half of Chicago's 112,057 crash records in 2024 are desk reports, made without an officer at the scene, and 91% of those involved no injury or tow. Crashes with an officer present are split roughly evenly between the two.

2. The damage threshold

Damage-only crashes are recorded with an estimated damage band:

print(C.damage.value_counts().to_string())
none = C.most_severe_injury == "NO INDICATION OF INJURY"
print(f"crashes with no injury: {none.mean():.1%}; of those, damage over $1,500: {(C.damage[none] == 'OVER $1,500').mean():.1%}")

damage
OVER $1,500      80378
$501 - $1,500    21357
$500 OR LESS     10322
crashes with no injury: 83.3%; of those, damage over $1,500: 70.6%

83.3% of the city's crashes injured no one, and most of those still caused over $1,500 of damage: a city crash record is mostly a property-damage record.

3. Who reports which roads

FARS records the agency responsible for the road. The unmatched FARS crashes show which roads the city records leave out:

print(M[~M.matched].RD_OWNERNAME.value_counts().to_string())
inter = M.FUNC_SYSNAME == "Interstate"
print(f"FARS Chicago crashes on interstates: {inter.sum()}; found in the city records: {M.matched[inter].sum()}")

RD_OWNERNAME
State Highway Agency                37
City or Municipal Highway Agency    18
County Highway Agency                2
Town or Township Highway Agency      1
FARS Chicago crashes on interstates: 25; found in the city records: 0

37 of the 58 unmatched FARS crashes are on roads owned by the state highway agency, which includes the expressways; none of the 25 interstate crashes is in the city's data.

Deaths in each crash source

The data used on this page

The City of Chicago's Traffic Crashes - Crashes (85ca-t3if): every crash in 2024, and crashes with a death between 31 December 2022 and 1 January 2024; and NHTSA's FARS 2023 crash file, crashes in the city of Chicago.

import zipfile
from pathlib import Path

import geopandas as gpd
import pandas as pd
import requests

Path("data").mkdir(exist_ok=True)
API = "https://data.cityofchicago.org/resource/85ca-t3if.json"


def socrata(name, **params):
    f = Path(f"data/{name}.parquet")
    if not f.exists():
        rows, offset = [], 0
        while True:
            part = requests.get(API, timeout=300, params={"$limit": 50_000, "$offset": offset, "$order": "crash_record_id", **params}).json()
            rows += part; offset += len(part)
            if len(part) < 50_000:
                break
        pd.DataFrame(rows).to_parquet(f)
    return pd.read_parquet(f)


C = socrata("chicago_crashes_2024", **{"$where": "crash_date >= '2024-01-01' AND crash_date < '2025-01-01'",
                                       "$select": "crash_record_id,crash_date,report_type,crash_type,damage,most_severe_injury"})
cf = socrata("chicago_fatal_crashes_2023", **{"$where": "crash_date >= '2022-12-31' AND crash_date < '2024-01-02' AND injuries_fatal > 0",
                                              "$select": "crash_record_id,crash_date,injuries_fatal,latitude,longitude"})
cf = cf.dropna(subset=["latitude", "longitude"]).astype({"latitude": float, "longitude": float})
CITY_FATAL = gpd.GeoDataFrame(cf.assign(date=pd.to_datetime(cf.crash_date).dt.normalize()),
                              geometry=gpd.points_from_xy(cf.longitude, cf.latitude), crs=4326).to_crs(3435)
F = Path("data/FARS2023NationalCSV.zip")
if not F.exists():
    F.write_bytes(requests.get("https://static.nhtsa.gov/nhtsa/downloads/FARS/2023/National/FARS2023NationalCSV.zip", timeout=600).content)
a = pd.read_csv(zipfile.ZipFile(F).open("FARS2023NationalCSV/accident.csv"), encoding="latin1", low_memory=False)
a = a[(a.STATE == 17) & (a.CITYNAME.str.upper() == "CHICAGO")]
FARS_CHI = gpd.GeoDataFrame(a.assign(date=pd.to_datetime(dict(year=a.YEAR, month=a.MONTH, day=a.DAY))),
                            geometry=gpd.points_from_xy(a.LONGITUD, a.LATITUDE), crs=4326).to_crs(3435)
print(f"Chicago crashes 2024: {len(C):,}; city fatal crashes around 2023: {len(CITY_FATAL)}; FARS 2023 Chicago crashes: {len(FARS_CHI)}")

Chicago crashes 2024: 112,057; city fatal crashes around 2023: 142; FARS 2023 Chicago crashes: 159

Example 1: deaths in each source

Count deaths, not crashes, for 2023 in both:

city23 = CITY_FATAL[CITY_FATAL.date.dt.year == 2023]
print(f"FARS 2023, Chicago: {FARS_CHI.FATALS.sum()} deaths in {len(FARS_CHI)} crashes")
print(f"city records 2023: {city23.injuries_fatal.astype(int).sum()} deaths in {len(city23)} crashes")
print(f"FARS deaths in crashes on interstates: {FARS_CHI.FATALS[FARS_CHI.FUNC_SYSNAME == 'Interstate'].sum()}")

FARS 2023, Chicago: 167 deaths in 159 crashes
city records 2023: 150 deaths in 142 crashes
FARS deaths in crashes on interstates: 27

FARS counts 167 deaths in Chicago in 2023 and the city records 150; the 27 deaths on interstates alone are more than the difference.

Each dataset filters by its purpose

Each crash dataset is a filter on all road crashes, set by its purpose. FARS exists to count road deaths nationally, so it applies a strict definition, death within 30 days, on a public road, involving a motor vehicle in transport, and gathers every such crash from every source available to state analysts, whichever agency attended. STATS19 exists to monitor injury collisions in Great Britain, so it takes every injury collision reported to the police and nothing else. A city's records exist to document what its own police department handled: every crash it reported, including minor damage-only ones, but not those handled by another force.

In Chicago the last point is concrete. The Illinois State Police patrol the expressways, so crashes there are not in the city's records, while FARS includes them. Every FARS 2023 crash on a Chicago interstate is missing from the city data, which is why a fatal-crash map from city data has empty expressways. The other unmatched crashes may differ in location or date between the two records, or be deaths days after a crash that the city recorded as an injury; the match cannot tell which. The lesson is not that one source is wrong, but that "crashes in Chicago" means three different things.

Desk reports, late deaths and under-reporting

• Desk reports: crashes reported later, without an officer at the scene, are mostly damage-only.
• Damage thresholds: many jurisdictions require a report only above a damage value, so damage-only counts are not comparable between places.
• Late deaths: a crash recorded as an injury and followed by a death within 30 days is fatal in FARS; the city record may not be updated.
• City name: FARS's CITYNAME follows place codes, which differ slightly from city boundaries; a spatial filter is an alternative.
• Under-reporting: injury crashes, especially those involving cyclists and pedestrians, are under-reported to the police everywhere; hospital data fill some of the gap.

Internal links

• How to build a high-injury network from crash data in Python — the city records
• Road injury severity scales explained — how injuries are graded
• Recorded crime data explained — the same question for crime records

FAQ

Why don't city crash data match FARS?

Because they cover different things: FARS counts every fatal crash on a public road, whichever agency handled it; city data cover what the city's police reported, often excluding expressways patrolled by state police.

Are expressway crashes in Chicago's crash data?

No: none of the 25 FARS 2023 fatal crashes on Chicago interstates appear in the city's records; the Illinois State Police handle them.

What is a desk report in crash data?

A crash reported to the police after the event, without an officer attending; in Chicago in 2024 they were half of all records, mostly without injury.

Does crash data include crashes without injuries?

Some does: Chicago's records include damage-only crashes; FARS includes only fatal crashes and STATS19 only injury collisions.