You cannot tell which data version produced a map

Problem statement

A figure in a January report says 27 countries have more than 50 million people. A rerun of the same pipeline in June says 29. Nobody changed the code — or nobody thinks they did — and there is no way to tell whether the source data was updated, a library changed a default, or a parameter was edited and committed as "tidy up".

This is the failure that every other guide in this cluster exists to prevent, and by the time you are asking the question the information has usually gone. This guide covers what can still be recovered, and how to make the next version answerable in a minute rather than a week. The examples replay that exact case with real data: a small project whose data/countries.geojson was Natural Earth's v4.1.0 release in January and was quietly replaced by v5.0.0 in June.

Quick answer

Work backwards through what is still on disk:

import subprocess, pathlib, datetime, geopandas as gpd

# 1. What does the artefact itself say?
data = gpd.read_file("data/countries.geojson")
print(data.crs, len(data), "| POP_YEAR:", data["POP_YEAR"].value_counts().head(2).to_dict())
from osgeo import gdal
print(gdal.OpenEx("data/countries.geojson").GetMetadata())    # VERSION? CONTENT_DATE?

# 2. When was the figure made, and what did the repository look like then?
made = datetime.datetime.fromtimestamp(pathlib.Path("reports/figure_3.png").stat().st_mtime)
print("figure made", made.date(), "| last commit before it:",
      subprocess.run(["git", "log", "-1", "--before", made.isoformat(), "--format=%h %ad %s",
                      "--date=short"], capture_output=True, text=True).stdout.strip())

# 3. Which inputs existed then, and have they changed since?
for p in pathlib.Path("data").glob("*"):
    print(p.name, datetime.datetime.fromtimestamp(p.stat().st_mtime).date(), f"{p.stat().st_size:,} bytes")

EPSG:4326 177 | POP_YEAR: {2019: 170, 2017: 3}
{}
figure made 2026-01-15 | last commit before it: 93b98f0 2026-01-10 Count countries over 50 million
countries.geojson 2026-06-02 813,456 bytes

The file carries no metadata of its own, but one of its columns is a version in all but name: POP_YEAR is 2019 for 170 of the 177 countries. The input's modification date is after the figure's. That is the hypothesis — the data was replaced — and the next steps test it.

File modification times and git log --before will usually narrow it to a day and a commit. That is enough to test a hypothesis; it is not enough to prove one.

Look for an embedded version, rerun and compare

1. Check the artefact for an embedded version

A GeoPackage may have a VERSION item, a GeoTIFF may have tags, a CSV may have a header comment. It is worth thirty seconds before anything else.

2. Compare the output against a rerun, feature by feature

If the difference is in a handful of features, it is a data change. If it is everywhere and small, it is a parameter or a library. If the schema changed, it is an upstream release.

a, b = gpd.read_file("reports/2026-01/big_countries.gpkg"), gpd.read_file("output/big_countries.gpkg")
print(len(a), "→", len(b), "rows |", len(a.columns), "→", len(b.columns), "columns | gone:",
      sorted(set(a.columns) - set(b.columns)))
print("added:  ", sorted(set(b["NAME"]) - set(a["NAME"])))
print("removed:", sorted(set(a["NAME"]) - set(b["NAME"])))

27 → 29 rows | 72 → 162 columns | gone: ['FIPS_10_', 'GDP_MD_EST', 'LASTCENSUS', 'WIKIPEDIA']
added:   ['Colombia', 'Kenya']
removed: []

Ninety-four new columns, four gone, and whole features added: an upstream release, not a library.

3. Test the data hypothesis

Re-download the input at the version you think was used, or find an archived copy, and run the current code against it. If the old number comes back, the data changed:

old = gpd.read_file("https://raw.githubusercontent.com/nvkelso/natural-earth-vector/"
                    "v4.1.0/geojson/ne_110m_admin_0_countries.geojson")
print("current code, v4.1.0 data:", (old["POP_EST"] > 50e6).sum(), "countries | POP_YEAR:",
      old["POP_YEAR"].value_counts().head(1).to_dict())

current code, v4.1.0 data: 27 countries | POP_YEAR: {2017: 172}

The January number comes back, and the archived release's POP_YEAR — 2017 — is the version the January figure should have carried in its caption.

4. Test the environment hypothesis

Recreate the environment at the date of the figure from the lock file, if one exists, and rerun. GEOS, PROJ and GDAL all change behaviour between releases, and a changed environment has its own signature. Here is one you can reproduce without installing anything — the same PROJ, with and without access to the OSTN15 grid it downloads for British National Grid:

import numpy as np
from pyproj import Transformer
from pyproj.network import set_network_enabled

places = gpd.read_file("https://raw.githubusercontent.com/nvkelso/natural-earth-vector/"
                       "v5.1.2/geojson/ne_10m_populated_places_simple.geojson")
uk = places[places["adm0name"] == "United Kingdom"]
xy = {}
for grid in (False, True):
    set_network_enabled(grid)
    xy[grid] = Transformer.from_crs(4326, 27700, always_xy=True).transform(uk.geometry.x.values, uk.geometry.y.values)
set_network_enabled(False)
shift = np.hypot(xy[True][0] - xy[False][0], xy[True][1] - xy[False][1])
print(f"{len(uk)} places, every one moved: {(shift > 0).all()} | "
      f"min {shift.min():.2f} m, median {np.median(shift):.2f} m, max {shift.max():.2f} m")

57 places, every one moved: True | min 0.23 m, median 1.75 m, max 4.03 m

Every feature moved, none by much, and the feature set is identical — the opposite of the data change above. A container rebuilt without proj-data, or on a machine that cannot reach PROJ's CDN, produces exactly this.

5. Test the code hypothesis

git log --since on the pipeline directory, looking for parameter changes rather than refactors. A diff restricted to numeric literals finds these fast:

git log -p --since=2026-01-15 -- pipeline/ | grep -E '^[-+].*[0-9]+\.?[0-9]*' | head -50

print(subprocess.run("git log -p --since=2026-01-15 -- pipeline/ | grep -E '^[-+][^-+].*[0-9]'",
                     shell=True, capture_output=True, text=True).stdout)

-threshold = 50_000_000
+THRESHOLD = 50_000_000

The only numeric line changed since the figure was made is the threshold, and the value is the same — the "tidy up" commit renamed threshold to THRESHOLD. The code hypothesis is ruled out.

6. Accept that it may be unanswerable, and fix the next one

If none of the above resolves it, say so in the report rather than guessing. Then make the next version answerable.

7. Put the version inside everything you publish from now on

The dataset, the figure, the report. A figure with a caption that ends natural-earth v5.0.0 · pipeline 3f9a1c2 is unambiguous forever, and costs one f-string.

8. Start recording lineage

How to record lineage automatically in a pipeline is the permanent fix, and it is about fifty lines.

Stamping figures and numbers

The data used on this page

Two real releases of Natural Earth's 1:110m countries (public domain), fetched from the tags of the project's GitHub repository, which do not change: v4.1.0, whose population estimates are for 2017, and v5.0.0, whose estimates are for 2019. The setup replays the story in the problem statement inside a real git repository — a pipeline committed in January, a figure made from v4.1.0, a "tidy up" commit in June, and the input swapped for v5.0.0 at the same path — with the commit dates and file times set to when each thing would have happened.

import os
import shutil
import subprocess
import sys
from datetime import datetime
from pathlib import Path

import requests

for d in (".git", "data", "output", "reports", "results", "pipeline"):
    shutil.rmtree(d, ignore_errors=True)
Path("pipeline").mkdir(); Path("data").mkdir()
Path(".gitignore").write_text("cell_*.py\n__pycache__/\ndata/\noutput/\nreports/\nresults/\n*.png\n")
PIPELINE = """import sys
from pathlib import Path
import geopandas as gpd
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt

threshold = 50_000_000

def main(out):
    countries = gpd.read_file("data/countries.geojson")
    big = countries[countries["POP_EST"] > threshold]
    Path(out).mkdir(parents=True, exist_ok=True)
    big.to_file(f"{out}/big_countries.gpkg")
    fig, ax = plt.subplots(figsize=(8, 4))
    countries.plot(ax=ax, color="#e2e8f0"); big.plot(ax=ax, color="#0ea5e9"); ax.set_axis_off()
    ax.set_title(f"{len(big)} countries with more than 50 million people")
    fig.savefig(f"{out}/figure_3.png", dpi=100)
    print(len(big))

main(sys.argv[1])
"""
Path("pipeline/count.py").write_text(PIPELINE)

def at(when):
    return {**os.environ, "GIT_AUTHOR_DATE": when, "GIT_COMMITTER_DATE": when,
            "GIT_AUTHOR_NAME": "Analyst", "GIT_AUTHOR_EMAIL": "analyst@example.org",
            "GIT_COMMITTER_NAME": "Analyst", "GIT_COMMITTER_EMAIL": "analyst@example.org"}

def git(*args, when="2026-01-10T09:00:00"):
    subprocess.run(["git", *args], check=True, capture_output=True, env=at(when))

def touch(path, when):
    t = datetime.fromisoformat(when).timestamp()
    for p in [Path(path), *Path(path).rglob("*")] if Path(path).is_dir() else [Path(path)]:
        os.utime(p, (t, t))

def release(tag):
    return requests.get("https://raw.githubusercontent.com/nvkelso/natural-earth-vector/"
                        f"{tag}/geojson/ne_110m_admin_0_countries.geojson", timeout=120).content

git("init", "-q", "-b", "main"); git("add", "."); git("commit", "-qm", "Count countries over 50 million")
Path("data/countries.geojson").write_bytes(release("v4.1.0")); touch("data/countries.geojson", "2026-01-14T16:20:00")
january = subprocess.run([sys.executable, "pipeline/count.py", "reports/2026-01"], capture_output=True, text=True).stdout.strip()
shutil.copy("reports/2026-01/figure_3.png", "reports/figure_3.png")
touch("reports", "2026-01-15T11:05:00")

Path("pipeline/count.py").write_text(PIPELINE.replace("threshold", "THRESHOLD"))
git("commit", "-qam", "tidy up", when="2026-06-03T14:30:00")
Path("data/countries.geojson").write_bytes(release("v5.0.0")); touch("data/countries.geojson", "2026-06-02T08:45:00")
june = subprocess.run([sys.executable, "pipeline/count.py", "output"], capture_output=True, text=True).stdout.strip()
print("January report:", january, "| June rerun:", june)
print(subprocess.run(["git", "log", "--format=%h %ad %s", "--date=short"], capture_output=True, text=True).stdout)

January report: 27 | June rerun: 29
b58380a 2026-06-03 tidy up
93b98f0 2026-01-10 Count countries over 50 million

Example 1 — stamp every figure

import matplotlib.pyplot as plt, subprocess, datetime

def provenance_stamp(dataset_version, extra=""):
    try:
        commit = subprocess.check_output(["git", "rev-parse", "--short", "HEAD"],
                                         text=True, stderr=subprocess.DEVNULL).strip()
        dirty = "+" if subprocess.check_output(["git", "status", "--porcelain"],
                                               text=True, stderr=subprocess.DEVNULL).strip() else ""
    except Exception:
        commit, dirty = "nogit", ""
    date = datetime.date.today().isoformat()
    return f"{dataset_version} · {commit}{dirty} · {date}{(' · ' + extra) if extra else ''}"

fig, ax = plt.subplots()
ax.set_title("29 countries with more than 50 million people")
stamp = provenance_stamp("natural-earth v5.0.0")
fig.text(0.99, 0.005, stamp, ha="right", va="bottom", fontsize=5.5, color="#666666")
fig.savefig("reports/figure_3_stamped.png", dpi=200)
print(stamp)

Path("pipeline/count.py").write_text(Path("pipeline/count.py").read_text() + "\n# a local edit\n")
print(provenance_stamp("natural-earth v5.0.0"))
subprocess.run(["git", "checkout", "--", "pipeline/count.py"], check=True)

natural-earth v5.0.0 · b58380a · 2026-09-30
natural-earth v5.0.0 · b58380a+ · 2026-09-30

The + for a dirty tree is the detail that matters: a commit hash from an uncommitted working tree identifies nothing, and marking it is more honest than omitting it.

Example 2 — stamp the numbers, not only the pictures

import json, pathlib

def write_result(name, value, dataset_version, **context):
    record = {"name": name, "value": value, "dataset_version": dataset_version,
              "stamp": provenance_stamp(dataset_version), **context}
    path = pathlib.Path("results") / f"{name}.json"
    path.parent.mkdir(exist_ok=True)
    path.write_text(json.dumps(record, indent=2))
    return value

big = write_result("countries_over_50m", 29, "natural-earth v5.0.0", threshold=50_000_000)
print(pathlib.Path("results/countries_over_50m.json").read_text())

{
  "name": "countries_over_50m",
  "value": 29,
  "dataset_version": "natural-earth v5.0.0",
  "stamp": "natural-earth v5.0.0 \u00b7 b58380a \u00b7 2026-09-30",
  "threshold": 50000000
}

A report generated from these files cannot quote a number without its provenance, because the number and the stamp come out of the same object.

Example 3 — find the candidate commits for a figure

import subprocess, pathlib, datetime

def commits_around(figure_path, pipeline_dir="pipeline", days=3):
    made = datetime.datetime.fromtimestamp(pathlib.Path(figure_path).stat().st_mtime)
    since = (made - datetime.timedelta(days=days)).isoformat()
    until = (made + datetime.timedelta(days=1)).isoformat()
    out = subprocess.run(
        ["git", "log", "--since", since, "--until", until, "--oneline", "--", pipeline_dir],
        capture_output=True, text=True).stdout
    return {"figure_made": made.isoformat(timespec="seconds"), "commits": out.splitlines()}

print(commits_around("reports/figure_3.png"))
print(commits_around("reports/figure_3.png", days=200))

{'figure_made': '2026-01-15T11:05:00', 'commits': []}
{'figure_made': '2026-01-15T11:05:00', 'commits': ['93b98f0 Count countries over 50 million']}

With the default three-day window it finds nothing: the last commit before the figure was five days earlier. Widening the window finds it. Modification times are weak evidence — a copy resets them — but they are usually the only evidence left, and they narrow the search to a handful of commits.

Three things feed a result; one is your code

Why "nobody changed the code" is usually true and irrelevant

Three things feed a result and only one of them is your code. The input file at a stable path is replaced on a schedule you do not control; the environment is rebuilt whenever a container is; and the parameters may live in a config file that is not reviewed as carefully as the code. The common case is a data refresh, and the reason it is hard to confirm is that nothing recorded which version was read.

Why library versions produce this exact symptom

A change in GEOS noding, a new default in a PROJ transformation pipeline, or a different resampling default changes many features by a small amount. That signature — everything moves slightly, nothing moves much — is almost diagnostic, and the test is to pin the old versions and rerun.

Why modification times are weak but worth using

mtime is set by the last write, and a copy, a sync or a restore resets it. It is still the most common surviving evidence, and combined with git log --before it usually narrows the search enough to test a hypothesis directly.

Why the caption is the highest-value fix

Everything else in this cluster requires infrastructure. A provenance stamp in a figure's corner requires one function and appears on every artefact that leaves the building. It does not tell you why a number changed, but it tells you exactly which inputs and which code produced it, which is the question that is otherwise unanswerable.

Notebook outputs, images and object versions

• Notebook outputs are evidence. A committed .ipynb records numbers and sometimes versions.
• pip freeze in the container image is the environment record if nothing else exists.
• Cloud storage keeps old versions. Object versioning may still hold the input you need.
• Check the config, not only the code. A YAML parameter change is a code change.
• Compare bounding boxes. An input whose extent grew is a re-release.
• Check the CRS. A silently changed CRS produces plausible, wrong numbers.
• Figure metadata can hold the stamp. PNG text chunks and PDF metadata both work.
• Say "unknown" in the report. A guessed provenance is worse than an acknowledged gap.

Internal links

• How to record lineage automatically in a pipeline — the permanent fix
• Provenance and lineage explained for spatial workflows — what to record and why
• Dataset versioning explained — the identifier the stamp quotes
• How to checksum spatial datasets so you can prove they match — confirming an input changed
• Reproducible GIS environments explained — pinning the libraries
• How to build a reproducible GIS workflow in Python — the wider practice
• How to store metadata inside a GeoPackage — putting the version in the file
• Batch jobs give different results each run — the same problem inside one pipeline

FAQ

How do I find out which data version produced an old map?

Check the artefact for an embedded version, then use the figure's modification time with git log --before to find the code, then compare the current inputs' dates and sizes against that. It narrows it to a day; it rarely proves it.

The numbers changed and the code did not. What happened?

Usually the input data was refreshed at a stable path, or the environment was rebuilt with newer versions of GEOS, PROJ or GDAL. Both produce small differences across many features.

How can I tell a data change from a library change?

A data change shows as added, removed or altered features. A library change shows as many features differing by a small amount with the feature set unchanged.

What is the cheapest thing I can do today?

Stamp every figure and every published number with the dataset version, the commit hash and the run date. One function, and it appears on everything.

Do file modification times prove anything?

No. They are reset by copies, syncs and restores. They are useful for narrowing a search and not for concluding one.

What if I genuinely cannot reconstruct it?

Say so in the report. An acknowledged gap is defensible; a guess presented as provenance is not.