How to Download Air Quality Station Data in Python

On this page

Problem statement

European air quality measurements are open, hourly and well documented, and both of the main sources need unpacking before they are a table you can analyse. The UK's AURN publishes one CSV per site and year, with a six-line header, three columns per pollutant and timestamps that run from 01:00 to 24:00. The European Environment Agency serves one Parquet file per sampling point through an API that accepts a date range and returns the whole history anyway, with values stored as text and a validity code on every row.

None of it is hard. All of it is silent when done wrong: pd.to_datetime refuses 24:00, filtering validity on the wrong code drops a third of a rural station's year, and a forgotten date filter averages thirteen years into "2023".

Quick answer

Two loaders that return the same tidy shape — one row per site, pollutant and hour, in UTC:

import io, requests, numpy as np, pandas as pd

def load_aurn(site, year):
    """UK AURN site-year → long table. Times are GMT, labelled at the END of each hour."""
    text = requests.get(f"https://uk-air.defra.gov.uk/datastore/data_files/site_data/"
                        f"{site}_{year}.csv", timeout=120).text
    d = pd.read_csv(io.StringIO(text), skiprows=4, low_memory=False)
    # "24:00" is midnight at the end of the day: build the timestamp from the day plus hours.
    hours = d["time"].str.slice(0, 2).astype(int)
    end = pd.to_datetime(d["Date"], dayfirst=True) + pd.to_timedelta(hours, unit="h")
    start = (end - pd.Timedelta("1h")).dt.tz_localize("UTC")
    out = []
    for i, col in enumerate(d.columns):
        if col in ("Date", "time") or col.startswith(("status", "unit")):
            continue
        out.append(pd.DataFrame({"site": site, "pollutant": col, "start": start,
                                 "value": pd.to_numeric(d[col], errors="coerce"),
                                 "status": d.iloc[:, i + 1], "unit": d.iloc[:, i + 2]}))
    return pd.concat(out, ignore_index=True)

aurn = load_aurn("MY1", 2023)
no2 = aurn[aurn.pollutant == "Nitrogen dioxide"]
print(f"{aurn.pollutant.nunique()} pollutants, {len(aurn):,} rows")
print(f"NO2: first hour starts {no2.start.min()}, last {no2.start.max()}, "
      f"{no2.value.notna().mean():.1%} captured, units {no2.unit.dropna().unique().tolist()}")

44 pollutants, 385,440 rows
NO2: first hour starts 2023-01-01 00:00:00+00:00, last 2023-12-31 23:00:00+00:00, 95.4% captured, units ['ugm-3']

Labelling each value by the start of its hour makes AURN data line up with most other hourly sources, and it moves the awkward 24:00 rows onto 23:00 of the same day.

Skip the preamble, rebuild timestamps, unpivot

1. Skip the preamble and read the site name

AURN files open with provenance lines and the site name on line four. skiprows=4 lands on the header; keep the name for labelling.

2. Rebuild timestamps from date plus hour

24:00 is not a valid clock time for pandas. Parse the date, add the hour as a timedelta, and subtract an hour if you want start-of-interval labels.

3. Unpivot the value/status/unit triplets

Each pollutant occupies three adjacent columns. A long table — site, pollutant, time, value, status, unit — is what every later step expects.

4. For EEA data, filter the dates yourself

The download API's dateTimeStart and dateTimeEnd select which sampling points are returned, not which rows are in the files. Each Parquet file contains the station's full history.

5. Convert values to numbers and keep the validity code

Value arrives as an object column. Validity is 1 for valid, 2 for valid but below the detection limit, −1 for invalid and −99 for maintenance. Keep rows with validity above zero.

6. Join the station metadata

Coordinates, station type and area are in a separate tab-separated file. Join on the sampling point identifier.

Two open sources, one tidy shape — and a different trap in each.

EEA Parquet files and AURN downloads

Example 1 — EEA Parquet files for a city

API = "https://eeadmz1-downloads-api-appservice.azurewebsites.net/ParquetFile/urls"

def load_eea(country, city, pollutant, year):
    r = requests.post(API, json={"countries": [country], "cities": [city],
                                 "pollutants": [pollutant], "dataset": 2,
                                 "dateTimeStart": f"{year}-01-01T00:00:00Z",
                                 "dateTimeEnd": f"{year}-12-31T23:00:00Z",
                                 "aggregationType": "hour"}, timeout=120)
    urls = [u.strip() for u in r.text.splitlines()[1:] if u.strip()]
    parts = [pd.read_parquet(io.BytesIO(requests.get(u, timeout=120).content)) for u in urls]
    return pd.concat(parts, ignore_index=True)

raw = load_eea("DE", "Berlin", "NO2", 2023)
print(f"{raw.Samplingpoint.nunique()} sampling points, {len(raw):,} rows, "
      f"from {raw.Start.min():%Y-%m-%d} to {raw.Start.max():%Y-%m-%d}")
print(f"Value dtype: {raw.Value.dtype}; units: {raw.Unit.unique().tolist()}")
print("validity codes:", raw.Validity.value_counts().sort_index().to_dict())

18 sampling points, 1,849,536 rows, from 2013-01-01 to 2025-12-31
Value dtype: object; units: ['ug.m-3']
validity codes: {-99: 1882, -1: 20407, 1: 1728289, 2: 98958}

Thirteen years for a request that asked for one. A mean over this frame is a mean over the station's history, and it will look entirely reasonable.

Example 2 — the year, the validity and the capture

raw["value"] = pd.to_numeric(raw.Value, errors="coerce")
year = raw[(raw.Start >= "2023-01-01") & (raw.Start < "2024-01-01")]
valid = year[year.Validity > 0]
per_point = valid.groupby("Samplingpoint").agg(n=("value", "size"), mean=("value", "mean"))
per_point["capture"] = per_point.n / 8760
print(f"{len(per_point)} points with 2023 data; "
      f"{(per_point.capture < 0.9).sum()} below 90% capture")
print(f"history-wide mean {raw[raw.Validity > 0].value.mean():.1f} µg/m³ "
      f"vs 2023 mean {valid.value.mean():.1f} µg/m³")

17 points with 2023 data; 0 below 90% capture
history-wide mean 24.0 µg/m³ vs 2023 mean 16.9 µg/m³

NO2 has fallen steadily over the decade, so the history-wide mean — 24.0 µg/m³ — overstates the 2023 level of 16.9 by 42%. The error has a direction and it is never flagged.

Example 3 — one tidy table with coordinates

meta = pd.read_csv("https://discomap.eea.europa.eu/map/fme/metadata/PanEuropean_metadata.csv",
                   sep="\t", low_memory=False,
                   storage_options={"User-Agent": "spatialworkflow-examples/1.0"})   # EEA refuses Python's default agent
valid = valid.assign(SamplingPoint=valid.Samplingpoint.str.split("/").str[1])
print("time zones declared for these points:",
      meta[meta.SamplingPoint.isin(valid.Samplingpoint.str.split("/").str[1])]
      .Timezone.unique().tolist())
tidy = valid.merge(meta[["SamplingPoint", "AirQualityStationEoICode", "AirQualityStationType",
                         "AirQualityStationArea", "Longitude", "Latitude"]]
                   .drop_duplicates("SamplingPoint"), on="SamplingPoint", how="left")
tidy = tidy.rename(columns={"Start": "start", "AirQualityStationEoICode": "station"})
tidy["start"] = tidy.start.dt.tz_localize("Etc/GMT-1").dt.tz_convert("UTC")
tidy = tidy.sort_values(["station", "start"], ignore_index=True)    # the download's file order varies
print(tidy[["station", "AirQualityStationType", "start", "value"]].head(3).to_string(index=False))
print(f"rows without coordinates after the join: {tidy.Longitude.isna().sum()}")
tidy.to_parquet("berlin_no2_2023.parquet")

time zones declared for these points: ['http://dd.eionet.europa.eu/vocabulary/aq/timezone/UTC+01']
station AirQualityStationType                     start  value
DEBE010            background 2022-12-31 23:00:00+00:00  18.86
DEBE010            background 2023-01-01 00:00:00+00:00   8.61
DEBE010            background 2023-01-01 01:00:00+00:00   5.45
rows without coordinates after the join: 0

EEA timestamps are in the time zone the metadata declares — UTC+01 all year for Germany, with no summer time — so they need localising to a fixed offset, not to Europe/Berlin, which would apply summer time the data does not use.

Validity 2 is valid data below the detection limit — common at clean sites, rare at kerbs.

Hour-ending labels and full histories

Why hour-ending labels exist

Instruments report the average of the hour just completed, so the natural label is the end of the hour. It is also why the last value of a day is 24:00: it belongs to that day. Converting to start-of-hour labels is a relabelling, not a shift in the data, and it makes joins with weather data straightforward.

Why the EEA files hold the full history

The download service is built around sampling points: the request selects which ones have data in the window, then serves their pre-built files. Filtering the rows is the client's job. That design makes the service fast and cacheable and makes this mistake easy.

Why validity 2 must be kept

Below the detection limit is still a measurement — the air was clean. Dropping those hours removes exactly the cleanest part of the record, which inflates annual means at rural and suburban stations and makes them look closer to urban ones than they are.

Three lines that make hour-ending data join cleanly with everything else.

Provisional data and multiple instruments

Provisional data: AURN marks recent data P; values can change on ratification months later.
Several instruments: an EEA station can have more than one sampling point for a pollutant; do not average them blindly.
UK sites in the EEA data: AURN data also appears in the EEA download after reporting, with a delay.
Daily files: the EEA service can return daily aggregates (aggregationType: "day"), which avoids the hourly volume for long periods.
Metadata file size: the EEA metadata table has tens of thousands of rows; cache it locally.

Internal links

Air pollutants and their units explained — what the columns measure
Air quality monitoring networks explained — what the station types mean
Air quality data gaps and negative values — validity, capture and artefacts in depth
How to analyse an air quality time series — the next step with the tidy table
How to compute an air quality index in Python — indices from the same data
Solar timestamps shifted by an hour — the same labelling problem in solar data

FAQ

Where can I download UK air quality data?

From UK-AIR: one CSV per AURN site and year at uk-air.defra.gov.uk/datastore/data_files/site_data/{SITE}_{YEAR}.csv.

Where can I download European air quality data?

From the EEA's Air Quality Download Service, which returns Parquet files per sampling point, and its station metadata file.

How do I handle 24:00 in the timestamps?

Parse the date and add the hour as a timedelta. 24:00 then becomes midnight at the end of that day.

Why does my EEA data contain years I did not ask for?

The date range selects sampling points; the files contain their full history. Filter the rows yourself.

Which validity codes should I keep?

Codes above zero: 1 (valid) and 2 (valid, below the detection limit).

What time zone are EEA timestamps in?

Local standard time of the reporting country, with no daylight saving. Localise to a fixed offset.