How to Find and Download Scanned Historical Maps in Python
On this page
Problem statement
You need an old map as a starting point — the Ordnance Survey six-inch sheet of a town in the 1880s, or a county map from the 1600s — and you want it as a full-resolution image you can georeference, not a screenshot of a viewer. Most map libraries show their scans through zoomable web viewers; the file behind the viewer is somewhere else, sometimes in several formats, and the licence is written in a place you have to go looking for.
Wikimedia Commons is one of the most useful sources because it holds large public-domain collections with an open API. This page finds the Ordnance Survey six-inch maps of Rutland surveyed in 1884–1891, reads their metadata, picks the sheet that covers Oakham, and downloads it — with the two things that go wrong on the way: the request refused for having no proper User-Agent, and the choice between a 57 MB TIFF and a 2.7 MB JPEG of the same image.
Quick answer
Search the File namespace, read imageinfo for the URL, size and licence, and download with a User-Agent that identifies you:
import requests
API = "https://commons.wikimedia.org/w/api.php"
UA = {"User-Agent": "spatialworkflow.io guide (https://www.spatialworkflow.io)"}
found = requests.get(API, headers=UA, timeout=60, params={
"action": "query", "list": "search", "srsearch": '"Map of Rutland OS Map name"',
"srnamespace": 6, "srlimit": 100, "format": "json"}).json()
titles = [hit["title"] for hit in found["query"]["search"]]
print(len(titles), "files:", sum(t.endswith(".jpg") for t in titles), "JPEG,",
sum(t.endswith(".tiff") for t in titles), "TIFF")
title = "File:Map of Rutland OS Map name 009-NW, Ordnance Survey, 1884-1891.jpg"
info = requests.get(API, headers=UA, timeout=60, params={
"action": "query", "titles": title, "prop": "imageinfo",
"iiprop": "url|size|sha1|extmetadata", "format": "json"}).json()
ii = next(iter(info["query"]["pages"].values()))["imageinfo"][0]
print(ii["width"], "x", ii["height"], "px,", round(ii["size"] / 1e6, 1), "MB,",
ii["extmetadata"]["LicenseShortName"]["value"])
Path("data/os_009-NW.jpg").write_bytes(requests.get(ii["url"], headers=UA, timeout=300).content)60 files: 30 JPEG, 30 TIFF
5343 x 3573 px, 2.7 MB, Public domain
Thirty quarter sheets of the county, each in two formats. The licence is in the metadata, not in the file name, and the download only works because the request says who is asking.
Search the file namespace, read metadata, choose the format
1. Search the File namespace, not the web pages
Commons is a MediaWiki, and every scan is a page in namespace 6, "File:". Searching that namespace with a quoted phrase from the collection's naming pattern returns the whole set; searching without srnamespace=6 returns galleries, categories and books that mention the words.
import re
sheets = sorted({re.search(r"name (\S+),", t).group(1) for t in titles})
print(len(sheets), "sheets:", " ".join(sheets))30 sheets: 002-SE 003-SE 003-SW 004-NE 005-NE 005-NW 005-SE 005-SW 006-NE 006-NW 006-SE 006-SW 007-NW 007-SW 008-SE 009-NE 009-NW 009-SE 009-SW 010-NE 010-NW 010-SE 010-SW 012-NE 013-NE 013-NW 013-SE 013-SW 014-NW 015-NW
The six-inch survey cut each county sheet into four quarter sheets — NW, NE, SW and SE — so a complete county is a grid of them. Thirty are on Commons for Rutland, and the list has holes: there is no sheet 011 at all, and most numbers have only one or two of their four quarters. Check the coverage of a collection before you plan a study area around it.
2. Read the metadata before you download anything
imageinfo with extmetadata returns the licence, the credit line and the description that the uploader recorded. For these sheets the description names the places on each sheet, which is the quickest way to find the one you want:
meta = requests.get(API, headers=UA, timeout=60, params={
"action": "query", "titles": "|".join(t for t in titles if t.endswith(".jpg")),
"prop": "imageinfo", "iiprop": "url|size|extmetadata", "format": "json"}).json()
pages = {p["title"]: p["imageinfo"][0] for p in meta["query"]["pages"].values()}
strip = lambda html: re.sub(r"<[^>]+>", "", html)
for t, page in sorted(pages.items()):
em = page["extmetadata"]
if "Oakham" in strip(em["ImageDescription"]["value"]):
print(re.search(r"name (\S+),", t).group(1), "|", strip(em["ImageDescription"]["value"]))
print(" licence:", em["LicenseShortName"]["value"], "| credit:", strip(em["Credit"]["value"]))009-NW | OS Map name 009/NWBraunston, Egleton, Oakham, LE15 8, LE15 6
licence: Public domain | credit: http://www.british-history.ac.uk/os-1-to-10560/rutland/009/nw
Sheet 009-NW covers Braunston, Egleton and Oakham. The credit points at British History Online, which published the scans, and the licence field says public domain — the survey is well over a century old and the Crown copyright on it has long expired.
3. Choose the format by measuring it
Both formats are the same 5,343 × 3,573 pixel image. The TIFF is uncompressed and 22 times larger:
import numpy as np
from PIL import Image
tif_title = title.replace(".jpg", ".tiff")
tif_info = requests.get(API, headers=UA, timeout=60, params={
"action": "query", "titles": tif_title, "prop": "imageinfo", "iiprop": "url|size", "format": "json"}).json()
tif = next(iter(tif_info["query"]["pages"].values()))["imageinfo"][0]
Path("data/os_009-NW.tiff").write_bytes(requests.get(tif["url"], headers=UA, timeout=600).content)
a = np.asarray(Image.open("data/os_009-NW.tiff").convert("L"), dtype=int)
b = np.asarray(Image.open("data/os_009-NW.jpg").convert("L"), dtype=int)
diff = np.abs(a - b)
print(f"TIFF {tif['size'] / 1e6:.1f} MB, JPEG {pages[title]['size'] / 1e6:.1f} MB")
print(f"mean difference {diff.mean():.2f} grey levels; pixels more than 32 apart: {(diff > 32).mean():.4%}")TIFF 57.3 MB, JPEG 2.7 MB
mean difference 1.00 grey levels; pixels more than 32 apart: 0.0131%
The JPEG differs from the TIFF by one grey level on average, and only about one pixel in eight thousand by more than a slight shade. For georeferencing — where you read the position of a church or a road junction — that is invisible. The TIFF is worth having for archival work or for automatic feature extraction that thresholds the ink; for everything on these pages the JPEG is the sensible download.
The two files also have unrelated URLs: Commons stores every file under a hash of its name, so you cannot turn a JPEG link into a TIFF link by changing the extension. Look each one up.
4. Download with a User-Agent that names you
Wikimedia's servers refuse requests that do not identify themselves:
anonymous = requests.get(ii["url"], timeout=60)
print(anonymous.status_code, anonymous.text[:90])403 Please set a user-agent and respect our robot policy https://w.wiki/4wJS. See also https:/
requests sends python-requests/2.x by default, which the policy treats as anonymous. A User-Agent with a project name and a URL or contact address is all it takes.
5. Use thumbnails for previews
iiurlwidth asks the server for a scaled copy, which is how the contact sheet at the top of this page was made — thirty previews for a few hundred kilobytes instead of thirty full sheets:
small = requests.get(API, headers=UA, timeout=60, params={
"action": "query", "titles": title, "prop": "imageinfo", "iiprop": "url",
"iiurlwidth": 1200, "format": "json"}).json()
th = next(iter(small["query"]["pages"].values()))["imageinfo"][0]
print(th["thumbwidth"], "x", th["thumbheight"], th["thumburl"][:60] + "…")1200 x 802 https://thumb.wikimedia.org/wikipedia/commons/thumb/a/a0/Map…
6. Keep the provenance next to the file
A scan without its source and licence is an orphan the moment it leaves the folder. Write the metadata you just read into a sidecar:
provenance = {"file": "os_009-NW.jpg", "commons_title": title, "source_url": ii["url"].split("?")[0],
"sha1": ii["sha1"], "width": ii["width"], "height": ii["height"],
"licence": ii["extmetadata"]["LicenseShortName"]["value"],
"credit": strip(ii["extmetadata"]["Credit"]["value"]),
"description": strip(ii["extmetadata"]["ImageDescription"]["value"])}
assert hashlib.sha1(Path("data/os_009-NW.jpg").read_bytes()).hexdigest() == ii["sha1"]
Path("data/os_009-NW.json").write_text(json.dumps(provenance, indent=2))
print(json.dumps(provenance, indent=1)[:300]){
"file": "os_009-NW.jpg",
"commons_title": "File:Map of Rutland OS Map name 009-NW, Ordnance Survey, 1884-1891.jpg",
"source_url": "https://upload.wikimedia.org/wikipedia/commons/a/a0/Map_of_Rutland_OS_Map_name_009-NW%2C_Ordnance_Survey%2C_1884-1891.jpg",
"sha1": "5dc6e4f103ab64b10d6217b255a396
The URL the API returns now carries tracking parameters (?utm_source=…); store it without them. The SHA-1 that Commons reports doubles as a download check: if the file on disk hashes to it, the download is complete and unmodified.
Downloading Rutland's six-inch sheets with provenance
The data used on this page
The Ordnance Survey six-inch (1:10,560) maps of Rutland, surveyed 1884–1891, as uploaded to Wikimedia Commons from British History Online; public domain. Nothing else is needed: every block on this page talks to the Commons API.
import hashlib
import json
from pathlib import Path
import requests
Path("data").mkdir(exist_ok=True)
print("ready")ready
Example 1: download every sheet of a collection, with provenance
def download_collection(query, folder, ext=".jpg"):
folder = Path(folder)
folder.mkdir(parents=True, exist_ok=True)
hits = requests.get(API, headers=UA, timeout=60, params={
"action": "query", "list": "search", "srsearch": query, "srnamespace": 6,
"srlimit": 100, "format": "json"}).json()["query"]["search"]
wanted = sorted(h["title"] for h in hits if h["title"].endswith(ext))
report = []
for i in range(0, len(wanted), 40):
batch = requests.get(API, headers=UA, timeout=60, params={
"action": "query", "titles": "|".join(wanted[i:i + 40]), "prop": "imageinfo",
"iiprop": "url|size|sha1|extmetadata", "format": "json"}).json()
for page in batch["query"]["pages"].values():
ii = page["imageinfo"][0]
report.append({"title": page["title"], "url": ii["url"], "bytes": ii["size"],
"licence": ii["extmetadata"]["LicenseShortName"]["value"]})
return report
report = download_collection('"Map of Rutland OS Map name"', "data/rutland")
print(len(report), "files,", round(sum(r["bytes"] for r in report) / 1e6), "MB in total;",
"licences:", sorted({r["licence"] for r in report}))30 files, 80 MB in total; licences: ['Public domain']
The function stops short of fetching the files so the page does not download 80 MB; add a loop over report with the same requests.get(..., headers=UA) to do so. Checking the licence of every file, not just the first, matters for collections assembled by many uploaders.
Example 2: find sheets by place name
def sheets_mentioning(place, pages):
return sorted(re.search(r"name (\S+),", t).group(1) for t, ii in pages.items()
if place.lower() in strip(ii["extmetadata"]["ImageDescription"]["value"]).lower())
for place in ("Oakham", "Hambleton", "Empingham", "Uppingham"):
print(f"{place:10}", sheets_mentioning(place, pages))Oakham ['009-NW']
Hambleton ['009-NE']
Empingham ['010-NW']
Uppingham ['013-NW']
Upper Hambleton and Empingham sit on the two sheets east of Oakham — the valley that was flooded in the 1970s to make Rutland Water, which is why these sheets are worth having: they show the landscape before it.
How scanned maps reach the web
Scanned maps reach the web through three layers: the institution that owns the paper, the project that scanned it, and the site that serves it. Each adds its own record, and the useful facts — what the map is, how big the scan is, and what you may do with it — are split between them. For the Rutland sheets the paper is the Ordnance Survey's, the scans came through British History Online, and Commons serves them with an API and a licence field.
An API matters more than it seems. A viewer shows you tiles at whatever zoom you ask for; a georeferencing workflow needs the original pixels, because the control points you place are only as precise as the image you place them on. Commons hands out the original through imageinfo and scaled copies through iiurlwidth, so the same few lines serve both previews and full downloads.
Format is a measurement, not a preference. A lossy JPEG sounds like a poor choice for a map, but a scan of black linework on white paper compresses well: the difference from the uncompressed TIFF is close to the scanner's own noise. Measure it once for the collection you use, and keep the TIFF only where the extra fidelity pays — thresholding ink into vectors is the usual case.
Finally, provenance is part of the data. The sidecar with the source URL, checksum and licence is what lets someone else — or you, next year — trust that the georeferenced map came from this scan and may be published.
Incomplete collections, licences and large originals
- Not every collection is complete. Commons holds 30 of the Rutland quarter sheets; others live only on the original library's site.
- Same name, different file. A JPEG and a TIFF of one sheet have unrelated URLs; Commons hashes file names into its paths.
- Licences vary within a category. Public domain for an 1880s survey does not make a 1950s revision free to reuse; check each file.
- Large originals. Some scans are hundreds of megapixels; request a thumbnail first to check you have the right sheet.
- Rate limits. The API is generous for batched queries — ask for up to 50 titles at once — and less so for one request per file.
- Descriptions are uploader text. Place lists like "Braunston, Egleton, Oakham" are helpful but not authoritative; confirm on the image.
- Library viewers. Many map libraries publish IIIF manifests; the same idea — read metadata, then fetch full-size images — applies there.
Internal links
- Georeferencing explained — what happens to the scan next
- How to pick ground control points for a scanned map — finding the same places on the old map and today
- How to georeference a scanned map with GDAL — from control points to a GeoTIFF
- How to record a dataset's licence — the provenance habit, for any download
- How to measure landscape change between an old map and today — what these sheets are for
FAQ
Where can I find scanned historical maps with an API?
Wikimedia Commons has an open MediaWiki API with licence metadata on every file. Many national and university libraries publish IIIF manifests for their map collections, which work the same way: read the metadata, then request the full-size image.
Why does my download return 403?
Wikimedia requires a User-Agent that identifies your project and a contact; the default python-requests/… string is refused with "Please set a user-agent and respect our robot policy". Set a descriptive one on every request.
Should I download the TIFF or the JPEG?
Measure both. For the 1880s OS sheets the JPEG differs from the TIFF by about one grey level and is 22 times smaller, which is plenty for georeferencing. Keep TIFFs for archives and for automatic extraction of linework.
How do I know a scan may be reused?
Read LicenseShortName and Credit in extmetadata, and check them for every file, not just one. The age of the map matters, but so does the scan: some institutions claim rights in their photographs of public-domain works.
How do I find which sheet covers a place?
Search the descriptions, which for many collections list place names, then confirm on a thumbnail. Sheet numbering schemes differ by county and by survey, so the number alone rarely tells you where a sheet is.