Finding & Fetching Data

Get the data before you analyse it — from OSM, web services and satellite catalogues.

The four stages of the finding-and-fetching-data path, from sources to a cache that makes a rerun free. where the data comes from, and how to stop fetching it twice1what existssources, and licences2OSM & Overpassthe biggest one3WFS & STACpaged, and throttled4cache it oncereruns cost nothingEvery source paginates, throttles or truncates differently — the last stage is what makes that survivable

Almost every tutorial starts with a file that already exists on disk. Real work starts earlier, with a question nobody has published the answer to: where does this layer come from, is it allowed to be used, and how do I get a current copy without clicking through a portal every month? These guides cover the four sources that answer most of it — OpenStreetMap, OGC web services, geocoders and STAC imagery catalogues — how each one paginates, throttles and silently truncates, and how to cache what you fetch so a rerun costs nothing.

Start at stage one if you are not sure what is available for your area — the first two guides are about sources and licences rather than code. If you already know you need OpenStreetMap, jump to stage two and come back for the caching guide at the end, which is the one that stops a monthly rerun costing an hour.

Start here

The cornerstone guide — the whole workflow end to end. Everything after it goes deeper into one part of it.

Step 1 GIS Data Sources Explained: Where Spatial Data Comes From gis data sources Begin the path →

The full path, in order

The concepts underneath the work, then each task, then the errors that task produces. Work down, or jump to the step you need.

  1. The OpenStreetMap Data Model Explained for Python Users openstreetmap data model 🧭 Concept
  2. How to Download OpenStreetMap Data in Python with OSMnx download openstreetmap python ⚙️ How-To
  3. How to Query the Overpass API from Python overpass api python ⚙️ How-To
  4. OSMnx Download Fails, Hangs or Times Out osmnx timeout error 🔧 Fix
  5. Spatial Web Services Explained: WFS, WMS and OGC API Features wfs wms explained 🧭 Concept
  6. How to Download Data from a WFS Service in Python download wfs python ⚙️ How-To
  7. How to Download Administrative Boundaries in Python download administrative boundaries python ⚙️ How-To
  8. GeoJSON Downloaded from an API Is Empty or Truncated api geojson truncated 🔧 Fix
  9. How to Geocode Addresses in Python geocode addresses python ⚙️ How-To
  10. Geocoding Returns Wrong or Missing Coordinates geocoding wrong coordinates 🔧 Fix
  11. STAC Explained: How Satellite Imagery Catalogues Work stac explained 🧭 Concept
  12. How to Search and Download Satellite Imagery with STAC in Python download satellite imagery python ⚙️ How-To
  13. How to Cache Downloaded GIS Data So You Fetch It Once cache downloaded gis data ⚙️ How-To

Topics this path covers

A path is a route; a topic is everything the site has on a subject. Open a topic to see the guides this path skips.

Data sources & downloads Files & formats Getting started Geocoding & addresses Data cleaning Raster & rasterio

Other ways in

📚 All learning paths 🗂️ All topics ⚙️ How-To guides 🔧 Fixes