Analysis
Getting from a raw download to something you can put in front of someone.
This is the largest section on the site, and it is all one thing: getting from a raw download to something you can put in front of someone. Clean, join, aggregate, map.
The code for all four steps is short. The decisions underneath are the analysis. What did you drop, and by what rule? Did the row count change at the join, and was that right? Are you mapping a count or a rate, and over which denominator? Which class breaks, which projection, which colours? Every one of those changes the conclusion, and none of them is recorded anywhere unless you record it.
The topics run from cleaning and raster work through statistics, terrain, hydrology, movement and public transit to the cartography that decides what the finished thing says.
Topics in Analysis
Each topic page brings that subject's concepts, how-to guides and fixes onto one page.
- Data cleaning Turning a messy download into an analysis-ready dataset you can trust.
- Raster & rasterio Reading, sampling and converting raster data in Python.
- Maps & visualisation Static plots, choropleths, interactive maps, and exporting images people can use.
- Spatial statistics Density, clustering, hotspots and autocorrelation — measuring pattern rather than drawing it.
- Census & demographics Census boundaries and tables: joining them, turning counts into rates, carrying margins of error, and comparing years.
- Housing & property data House prices and short-term rentals in Python: HM Land Registry price paid data, Inside Airbnb listings and calendars, price maps by area, hedonic and repeat-sales indices, and the parsing and joining traps in both.
- Terrain & elevation Working with DEMs: slope, aspect, hillshade, contours, profiles and drainage.
- Hydrology & watersheds Where water goes on a DEM: conditioning, flow direction and accumulation, watersheds, stream networks and HAND.
- Movement & time GPS tracks and trajectories: cleaning them, segmenting them, and measuring what moved where.
- Public transit & GTFS Timetables as data: GTFS feeds, service calendars, frequency, transit travel times and the realtime feeds that correct them.
- Web maps & tiles Getting a dataset into a browser: tile pyramids, vector tiles, PMTiles and MapLibre styling.
- Python map libraries Making maps with the popular Python libraries: Plotly choropleths, scatter, density and animated maps, Folium heatmaps, clusters and tooltips, GeoPandas explore, Cartopy projections, pydeck 3D layers and lonboard for millions of points, with the errors each one throws.
- Cartography & map design The decisions that turn a correct plot into a map someone can read: hierarchy, colour, labels and export.
- Election maps & cartograms Mapping votes rather than land: county and constituency results, swing, cartograms, hex and tile maps, dot density and value-by-alpha.
- Animated & time-series maps Maps that change over time: radar and satellite loops, storm tracks, animated choropleths, small multiples, time sliders and the export formats that carry them.
- Wildfire mapping Fire perimeters, satellite fire detections, burn severity, smoke plumes and exposure: mapping wildfires from open CAL FIRE, NOAA GOES, HMS and Sentinel-2 data.
- Land cover & land change Land cover rasters and how they change: NLCD and ESA WorldCover downloads, change matrices, urban growth, impervious surface, class areas and categorical raster pitfalls.
- Health & disease mapping Mapping health: John Snow’s cholera map, CDC PLACES and life expectancy by tract, age-adjusted rates, overdose deaths, scan statistics, hospital access and the ecological fallacy.
- Flow maps & origin–destination data Movement between places: migration, commuting and trip flows, origin–destination matrices, desire lines, curved and bundled flow maps, and the uncertainty in small flows.
- Bike share & micromobility Bike-share trip files and live GBFS feeds: cleaning a year of trips, dockless e-bikes, commuting rhythms, weather, speeds and station coverage.
- Air travel & flight data A year of US domestic flights from the BTS on-time file: airports, routes, delays and their causes, taxi times, aircraft rotations and the jet stream.
- Road traffic counts Britain's road traffic counts in Python: annual average daily flows at 46,754 count points, how they are estimated, trends, hourly profiles, vehicle-kilometres, cycling, lorries and collision rates.
- Street trees Street tree inventories in Python, from New York's 2005 and 2015 censuses: species diversity, trees per kilometre, spacing gaps, health, change between censuses and volunteer data.
- Drought Drought data in Python: US Drought Monitor county statistics, the Standardized Precipitation Index from rain gauges, drought events, flash droughts, streamflow and the people affected.
- Taxi & ride-hail trips NYC taxi and ride-hail trip records in Python: zones, fares, tips, speeds, airport trips, congestion pricing, and the fields that mislead.
- Power plants & electricity US power plants in Python from EIA-860 and EIA-923: where plants are, what they generate, capacity factors, retirements, new solar and batteries, and the data traps.
- Bridges & infrastructure condition The US National Bridge Inventory in Python: mapping 624,193 highway bridges, condition ratings, age, traffic on poor bridges, scour, inspections and the format traps.
- 311 & service requests New York's 311 service requests in Python: complaint types, response times, noise, heat and hot water, maps per resident, and the duplicates and impossible dates in the records.
- Severe storms & tornadoes NOAA's Storm Events database in Python: tornado tracks, hail by county, weather deaths, damage estimates, the severe-weather season and the traps in damage strings, zone codes and county segments.
- Hurricanes & tropical cyclones NHC's HURDAT2 best tracks in Python: mapping hurricane seasons, accumulated cyclone energy, US landfalls, rapid intensification, wind radii and the coordinate and record-type traps.
- Disasters & flood insurance OpenFEMA in Python: disaster declarations by county since 1953, declaration types and programs, NFIP flood insurance claims, hurricane claim maps, and the per-county rows, statewide codes and negative payments that trip up a count.
- Electric vehicles Washington State's electric vehicle registrations in Python: adoption by county and census tract, growth since 2017, battery-electric against plug-in hybrids, makes and models, and the zero ranges, ZIP-centroid points and VIN prefixes in the data.
- Restaurant inspections New York's restaurant inspection results in Python: letter grades and scores, violations, closures, pests by neighbourhood, and the placeholder dates, per-violation rows and zero coordinates in the data.
- Historical maps & georeferencing Putting scanned old maps on today's grid: control points, transformations, warping, accuracy, and measuring what has changed since.
- Snow, ice & glaciers Snow cover from satellites, snow depth and water equivalent from stations, sea-ice extent, and glaciers measured from imagery and elevation models.
- Crime & public safety mapping Mapping recorded crime: city open data, hot spots and kernel density, rates per resident, time patterns, near repeats, street segments and what the records leave out.
- Road crashes & traffic safety Mapping road crashes: FARS and STATS19, fatal and injury collisions, rates per resident and per mile driven, pedestrians and cyclists, time of day and light, and the codes that hide missing values.
- Population grids Gridded population: GHSL, WorldPop and census grids, totals per area, density per square kilometre, change since 1975, degree of urbanisation, and the cell-area, resampling and suppression traps that change the count.
- Sea surface temperature & marine heatwaves Sea surface temperature from NOAA OISST and ERSST through ERDDAP: daily maps, anomalies and baselines, marine heatwaves, coral bleaching heat stress, El Niño indices, trends, buoys against satellites, and the longitude, packing and weighting traps.
- Earthquakes & seismic hazard Earthquakes as data from USGS: Gutenberg-Richter b-values and recurrence, ShakeMap stations and Did You Feel It? reports, ground motion against distance, faults, plate boundaries, focal mechanisms, slab depth, hazard curves and maps, and the unit, time and distance traps.
- Wind & vector fields Wind as data: GFS and HRRR winds from AWS, u and v components, speed and direction, arrows, barbs and streamlines, station observations, wind roses and vorticity.
- Rainfall, radar & storms Weather radar volumes, radar and multi-sensor rainfall grids, rain gauges, storm tracks, and how extreme a rainfall total really was.
- Coastal & marine Depths, tides and shorelines: bathymetry grids, which line is the coast, inundation, vessel tracks and the sea areas a country can claim.
- Agriculture & vegetation Fields, crops and canopies: boundaries, vegetation indices over a season, yield maps, management zones and prescription files.
- Biodiversity & species data Occurrence records as spatial data: what a GBIF record is, the bias in it, richness and diversity per cell, distribution models, ranges and protected areas.
- Linear referencing Locating things along a route rather than on the earth: measures and M values, postmiles and equations, dynamic segmentation, and moving events between networks.
- Natural hazards & risk Floods, fires, earthquakes and landslides as data: return periods, severity and shaking, who is exposed, and the arithmetic that turns the three into a risk figure.
- Air quality Pollution as data: station networks and their units, indices, satellite NO2, models, interpolation, land-use regression and population exposure.
- Renewable energy siting Sun, wind and land as data: irradiance and wind resource, yield models, exclusion masks, terrain shading, grid distance and suitability maps.
- Urban form & morphology Cities measured from their footprints and streets: coverage and floor area ratio, tessellation cells, blocks, street orientation and network metrics, and where the city ends.
- Forest monitoring Forest cover and loss as data: Global Forest Change, forest definitions and thresholds, rates, fragmentation, protected areas, hotspots, canopy height and new clearing.
- Urban heat Heat in cities as data: land surface temperature from Landsat and MODIS, heat islands by day and night, green space, air temperature, heatwaves and who is exposed.