Reproducible GIS Environments Explained: conda, pip, Lockfiles and Docker #

Problem statement #

The script worked in March. In August, on a fresh machine, it does not:

ImportError: libgdal.so.32: cannot open shared object file: No such file or directory

Or it imports fine and gives different numbers:

gdf.to_crs(27700).geometry.iloc[0].bounds
# March:  (351204.117, 381009.882, 351298.443, 381012.004)
# August: (351204.121, 381009.879, 351298.447, 381012.001)

Four millimetres. Nothing in the code changed. PROJ 9.2 shipped an updated OSTN15 transformation grid, and the two runs used different ones.

Python GIS has a reproducibility problem that pure-Python projects do not, because the important software is not Python. GeoPandas is a thin layer over GEOS, PROJ and GDAL — C and C++ libraries with their own versions, their own data files, and their own opinions. A requirements.txt pinning geopandas==1.0.1 says nothing about which GEOS it was compiled against.

Quick answer #

Pick the strongest level of pinning the project justifies:

# a solid default for a Python GIS project
pixi init && pixi add python=3.12 geopandas rasterio pyproj
pixi list                        # writes pixi.lock — commit it

# and for anything scheduled or shared
FROM ghcr.io/osgeo/gdal:ubuntu-small-3.9.2@sha256:1f9c...

The rule that matters most: the native stack — GEOS, PROJ, GDAL — must be pinned along with the Python packages, or you have not pinned anything that determines your results.

Step-by-step solution #

1. Find out what you are actually running #

import geopandas, shapely, pyproj, rasterio, fiona
from shapely import geos_version_string
import pyproj

print(f"geopandas  {geopandas.__version__}")
print(f"shapely    {shapely.__version__}   GEOS {geos_version_string}")
print(f"pyproj     {pyproj.__version__}    PROJ {pyproj.proj_version_str}")
print(f"rasterio   {rasterio.__version__}  GDAL {rasterio.__gdal_version__}")
print(f"fiona      {fiona.__version__}     GDAL {fiona.__gdal_version__}")
print(f"PROJ data  {pyproj.datadir.get_data_dir()}")
print(f"network    {pyproj.network.is_network_enabled()}")

geopandas  1.0.1
shapely    2.0.6   GEOS 3.12.1-CAPI-1.18.1
pyproj     3.6.1    PROJ 9.4.0
rasterio   1.3.10  GDAL 3.8.4
fiona      1.9.6   GDAL 3.8.4
PROJ data  /opt/conda/share/proj
network    True

The last two lines are the ones almost nobody records and both change results. PROJ data is where the transformation grids live, and network True means PROJ will download grids on demand — so the same code on the same versions can transform differently depending on whether the machine had internet and what was cached.

Two GDAL versions on the same line (rasterio and fiona) that disagree is a warning sign: you have two GDAL builds loaded, and which one wins depends on link order.

2. Understand why pip install geopandas is not enough #

A pinned requirements.txt fixes the Python versions and nothing else:

geopandas==1.0.1
shapely==2.0.6
pyproj==3.6.1

Install that in March and August and you can still get different GEOS, PROJ and GDAL, because:

• Wheels bundle their own native libraries. shapely ships a GEOS inside its wheel; pyproj ships PROJ; rasterio and fiona each ship GDAL. Which build you get depends on which wheel your platform resolved to.
• Several bundled copies coexist. A process can hold shapely's GEOS and GDAL's GEOS at once. This mostly works and is the source of the strangest bugs.
• PROJ's grid files are data, not code, and are versioned separately. pyproj bundles a minimal set and downloads the rest.
• A source build links against whatever is on the system, so the same pinned version compiles against a different GEOS on a different machine.

pip freeze records none of this. It is not that pinning is wrong — it is that it pins the layer that matters least.

3. Use a lockfile that covers the native stack #

conda packages the C libraries as first-class packages, so a conda lockfile pins GEOS, PROJ and GDAL alongside Python:

# environment.yml — the human-edited intent
cat > environment.yml <<'YAML'
name: gis
channels: [conda-forge]
dependencies:
  - python=3.12
  - geopandas=1.0.1
  - rasterio=1.3.10
  - pyproj=3.6.1
  - psycopg=3.2
  - pytest
YAML

# conda-lock — the machine-generated exact solution
conda-lock lock -f environment.yml -p linux-64 -p osx-arm64
conda-lock install --name gis conda-lock.yml

conda-lock.yml names every package, every version, every build string and a hash, for each platform. Commit it. Recreating it in two years gives the same GEOS, the same PROJ, the same PROJ data package.

pixi is the modern front end for the same solver and is simpler:

pixi init gis-project && cd gis-project
pixi add python=3.12 geopandas rasterio pyproj pytest
pixi run python -c "import geopandas; print(geopandas.__version__)"

pixi.toml holds the intent, pixi.lock holds the exact solution for every platform you declare. Both belong in version control.

If you must stay on pip, uv produces a hash-pinned lockfile:

uv pip compile requirements.in -o requirements.txt --generate-hashes
uv pip sync requirements.txt

This is genuinely reproducible for the Python layer and for the native libraries bundled inside wheels — which is most of the stack. What it cannot pin is a system GDAL that a source build would link against, or PROJ grid data.

4. Pin the PROJ data and turn off surprise downloads #

This is the step that catches the four-millimetre bug:

# make the grid set explicit and complete
conda install -c conda-forge proj-data     # the full transformation grid set

import os
os.environ["PROJ_NETWORK"] = "OFF"          # before importing pyproj

import pyproj
pyproj.network.set_network_enabled(False)   # or at runtime

print(pyproj.datadir.get_data_dir())
print(pyproj.network.is_network_enabled())  # False

With the network on, PROJ fetches grids from a CDN when a transformation needs one, caches them, and uses them. That is convenient and it means the same code gives different answers depending on network availability and cache state. For anything whose numbers matter, install proj-data and turn the network off, so a missing grid is an error rather than a silent fallback to a less accurate method.

Record which transformation was actually used:

from pyproj import Transformer

t = Transformer.from_crs(4326, 27700, always_xy=True)
print(t.description)

Inverse of OSGB36 to WGS 84 (9) + British National Grid

That string is worth logging next to results — it names the transformation pipeline, which is the thing that changed between March and August. See reprojecting between datums correctly.

5. Containerise when the environment must travel #

A lockfile reproduces an environment on a machine that can install packages. A container reproduces the machine.

FROM ghcr.io/osgeo/gdal:ubuntu-small-3.9.2

ENV PROJ_NETWORK=OFF \
    PYTHONDONTWRITEBYTECODE=1 \
    PIP_NO_CACHE_DIR=1

RUN apt-get update && apt-get install -y --no-install-recommends \
        python3-pip proj-data \
    && rm -rf /var/lib/apt/lists/*

COPY requirements.txt .
RUN pip install --no-cache-dir --require-hashes -r requirements.txt

COPY src/ /app/src/
WORKDIR /app

Note what each line buys. The base image supplies GDAL, GEOS and PROJ as one tested set — the hardest part to get right by hand. PROJ_NETWORK=OFF and proj-data make transformations deterministic. --require-hashes means a compromised or re-uploaded package fails the build rather than silently changing it.

The base image tag is still mutable. For a scheduled job, pin the digest:

FROM ghcr.io/osgeo/gdal:ubuntu-small-3.9.2@sha256:1f9c4e0a...

A tag can be re-pushed; a digest is the content hash and cannot. Details in how to build a small, fast Python GIS Docker image.

6. Record the environment with the results #

Whatever level you chose, write down what actually ran:

import json, platform, sys
from pathlib import Path

def environment_record():
    import geopandas, shapely, pyproj, rasterio, fiona
    from shapely import geos_version_string
    return {
        "python": sys.version.split()[0],
        "platform": platform.platform(),
        "geopandas": geopandas.__version__,
        "shapely": shapely.__version__,
        "geos": geos_version_string,
        "pyproj": pyproj.__version__,
        "proj": pyproj.proj_version_str,
        "proj_data_dir": str(pyproj.datadir.get_data_dir()),
        "proj_network": pyproj.network.is_network_enabled(),
        "rasterio": rasterio.__version__,
        "gdal": rasterio.__gdal_version__,
        "fiona_gdal": fiona.__gdal_version__,
    }

Path("run_environment.json").write_text(json.dumps(environment_record(), indent=2))

Writing this next to every output turns "it gives different numbers now" from a mystery into a diff. It is the environment half of recording run metadata and data lineage.

Code examples #

Example 1: a startup check that fails fast on the wrong environment #

import sys
from dataclasses import dataclass

@dataclass(frozen=True)
class Requirement:
    name: str
    minimum: tuple
    actual: tuple

def _tuple(v):
    return tuple(int(p) for p in str(v).split(".")[:3] if p.isdigit())

def check_environment(*, strict=True, require_offline_proj=True):
    import geopandas, shapely, pyproj, rasterio
    from shapely import geos_version_string

    geos = _tuple(geos_version_string.split("-")[0])
    checks = [
        Requirement("geopandas", (1, 0), _tuple(geopandas.__version__)),
        Requirement("shapely", (2, 0), _tuple(shapely.__version__)),
        Requirement("GEOS", (3, 10), geos),
        Requirement("pyproj", (3, 4), _tuple(pyproj.__version__)),
        Requirement("PROJ", (9, 0), _tuple(pyproj.proj_version_str)),
        Requirement("GDAL", (3, 6), _tuple(rasterio.__gdal_version__)),
    ]

    problems = [f"{c.name} {'.'.join(map(str, c.actual))} is below the required "
                f"{'.'.join(map(str, c.minimum))}"
                for c in checks if c.actual < c.minimum]

    if require_offline_proj and pyproj.network.is_network_enabled():
        problems.append("PROJ_NETWORK is ON — transformation grids may be fetched at "
                        "runtime, so results depend on network state. Set PROJ_NETWORK=OFF "
                        "and install proj-data.")

    import fiona
    if rasterio.__gdal_version__ != fiona.__gdal_version__:
        problems.append(f"rasterio links GDAL {rasterio.__gdal_version__} but fiona links "
                        f"{fiona.__gdal_version__} — two GDAL builds in one process")

    for c in checks:
        print(f"  {c.name:<10} {'.'.join(map(str, c.actual))}")
    for p in problems:
        print(f"  ✗ {p}")
    if problems and strict:
        sys.exit(f"environment check failed ({len(problems)} problem(s))")
    return problems

check_environment()

  geopandas  1.0.1
  shapely    2.0.6
  GEOS       3.12.1
  pyproj     3.6.1
  PROJ       9.4.0
  GDAL       3.8.4
  ✗ PROJ_NETWORK is ON — transformation grids may be fetched at runtime, so results
    depend on network state. Set PROJ_NETWORK=OFF and install proj-data.

The two-GDAL check catches a real and confusing condition: rasterio and fiona each bundle GDAL, and if their versions differ, one build wins per process by link order. Symptoms include a driver available to one library and not the other, and format support that changes depending on import order.

Running this at the top of a scheduled job turns an environment drift into an immediate, explanatory failure rather than a wrong number three steps later.

Example 2: a pinned, portable project layout #

gis-project/
├── pixi.toml                 # human intent
├── pixi.lock                 # exact solution, all platforms — COMMIT THIS
├── Dockerfile                # for scheduled runs
├── src/
│   └── pipeline/
├── tests/
└── environment_record.json   # what actually ran, written per run

# pixi.toml
[project]
name = "gis-project"
channels = ["conda-forge"]
platforms = ["linux-64", "osx-arm64", "win-64"]

[dependencies]
python = "3.12.*"
geopandas = "1.0.*"
rasterio = "1.3.*"
pyproj = "3.6.*"
proj-data = "*"                # the full transformation grid set
psycopg = "3.2.*"

[feature.dev.dependencies]
pytest = "*"
pytest-cov = "*"
ruff = "*"

[environments]
default = []
dev = ["dev"]

[activation.env]
PROJ_NETWORK = "OFF"

[tasks]
test = "pytest -m 'not slow'"
test-all = "pytest"
lint = "ruff check src tests"

pixi install                 # exactly what pixi.lock says
pixi run test
pixi run --environment dev lint

Three choices here are deliberate. platforms lists every OS anyone uses, so the lockfile solves for all of them and a Mac developer and a Linux CI runner get the same versions. proj-data is an explicit dependency rather than an assumption. And PROJ_NETWORK = "OFF" is set by the environment activation, so nobody has to remember it.

Version specifiers use 1.0.* rather than exact pins, because pixi.lock is what provides exactness — the .toml records intent, the lock records the solution. Editing the lock by hand defeats the point.

Example 3: proving reproducibility in CI #

An environment is only reproducible if you check:

# .github/workflows/reproducible.yml
name: reproducible
on: [push, pull_request]

jobs:
  matrix:
    strategy:
      fail-fast: false
      matrix:
        os: [ubuntu-latest, macos-latest, windows-latest]
    runs-on: ${{ matrix.os }}
    steps:
      - uses: actions/checkout@v4
      - uses: prefix-dev/setup-[email protected]
        with:
          locked: true              # fail if pixi.lock is out of date
      - name: Record the environment
        run: pixi run python -c "
          import json,sys,geopandas,shapely,pyproj,rasterio;
          from shapely import geos_version_string;
          print(json.dumps({'os':sys.platform,'gpd':geopandas.__version__,
          'geos':geos_version_string,'proj':pyproj.proj_version_str,
          'gdal':rasterio.__gdal_version__,
          'net':pyproj.network.is_network_enabled()}))" | tee env-${{ matrix.os }}.json
      - name: Golden-value transformation check
        run: pixi run pytest tests/test_reproducibility.py -v
      - uses: actions/upload-artifact@v4
        with:
          name: env-${{ matrix.os }}
          path: env-${{ matrix.os }}.json

# tests/test_reproducibility.py
import pytest
from pyproj import Transformer

# Recorded 2026-08-21 with PROJ 9.4.0, proj-data installed, PROJ_NETWORK=OFF.
GOLDEN = [
    # (from, to, x, y, expected_x, expected_y, tolerance_m)
    (4326, 27700, -2.2426, 53.4808, 383_618.507, 398_050.393, 0.001),
    (4326, 3857, -2.2426, 53.4808, -249_643.098, 7_073_628.702, 0.001),
    (27700, 4326, 383_618.507, 398_050.393, -2.2426, 53.4808, 1e-7),
]

@pytest.mark.parametrize("src,dst,x,y,ex,ey,tol", GOLDEN)
def test_transformation_is_stable(src, dst, x, y, ex, ey, tol):
    t = Transformer.from_crs(src, dst, always_xy=True)
    gx, gy = t.transform(x, y)
    assert gx == pytest.approx(ex, abs=tol), f"pipeline: {t.description}"
    assert gy == pytest.approx(ey, abs=tol), f"pipeline: {t.description}"

def test_proj_network_is_off():
    import pyproj
    assert not pyproj.network.is_network_enabled(), (
        "PROJ_NETWORK is on — transformation results depend on network state")

The golden-value test is the one that would have caught the four-millimetre drift. It pins actual numerical output of a datum transformation, not a version string, so any change in PROJ, in the grid data, or in the selected pipeline fails visibly.

Putting t.description in the assertion message is what makes the failure diagnosable: it names the transformation pipeline PROJ chose, so a failure says which transformation changed rather than only that something did.

locked: true makes CI fail if pixi.lock does not match pixi.toml, which prevents the common drift where someone adds a dependency and forgets to regenerate the lock.

Explanation #

Reproducibility in Python GIS is harder than in most Python work, and the reason is a single structural fact: the software that computes your answers is not written in Python.

GeoPandas dispatches to Shapely, which wraps GEOS. Reprojection goes through pyproj to PROJ. File I/O goes through fiona or pyogrio to GDAL/OGR. These are large C and C++ codebases with their own release cycles, their own bundled data, and behaviour that changes between versions in ways that are correct and still change your output. A pinned geopandas==1.0.1 constrains a wrapper. It says nothing about the code doing the work.

The data files are the part most often forgotten. PROJ's accuracy for a datum transformation depends on grid files — OSTN15 for Britain, NADCON for North America, and hundreds of others. These ship separately from PROJ itself, get corrected over time, and can be fetched over the network at runtime. A transformation with a grid available and the same one without differ by metres, not millimetres; and with PROJ_NETWORK=ON, whether a grid is available depends on the machine's internet access and cache. That is a dependency on network state hiding inside a coordinate transformation, and it is invisible in every version listing.

Each level of pinning closes one class of variation. Pinned versions close package drift. Hashes close the case where a version is re-uploaded with different content. A conda-family lockfile closes native library drift, because GEOS, PROJ and GDAL are packages in that ecosystem rather than opaque wheel payloads. A container closes OS-level variation — glibc, system libraries, locale. And a digest closes the fact that a container tag is itself mutable.

The right level is a cost decision rather than a technical one. Exploratory analysis needs none of this. A published result needs a lockfile. A scheduled job that must produce identical output for years needs a digest-pinned image. Choosing the strongest level for everything is expensive and slows people down; choosing the weakest for a production pipeline produces the August failure.

A last point about what reproducibility buys. The goal is not that numbers never change — libraries improve, and a more accurate transformation is a better one. The goal is that changes are visible and attributable. A golden-value test that fails when PROJ updates is doing its job: it converts a silent four-millimetre drift into a pull request with a decision attached. Recording the environment alongside results does the same for outputs already published. That is the property worth engineering for, and it is a great deal cheaper than never changing anything.

Edge cases or notes #

• Wheels bundle their own native libraries. shapely, pyproj, rasterio and fiona each ship a copy, so several GEOS or GDAL builds can be loaded at once.
• rasterio.__gdal_version__ != fiona.__gdal_version__ means two GDAL builds in one process; which wins depends on link order.
• PROJ_NETWORK=ON makes results depend on network state. Install proj-data and set it off for anything whose numbers matter.
• Transformer.description names the pipeline actually chosen — log it with results.
• pip freeze does not record native versions. Neither does conda list --export unless you use build strings.
• Container tags are mutable. Pin @sha256:… for scheduled work.
• Apple Silicon and x86 resolve to different wheels, so declare every platform in your lockfile.
• --require-hashes makes pip refuse anything whose content changed, which is the point of hashes.
• conda and pip in one environment is a common source of two incompatible native stacks. Prefer one, and if you must mix, install conda packages first.
• A lockfile you never restore from is not a lockfile. Have CI install from it, with locked: true or equivalent.

Internal links #

• How to make a GIS workflow reproducible in Python — the wider practice
• How to set up a Python GIS environment that actually works — getting one running in the first place
• How to build a small, fast Python GIS Docker image — the container level
• GDAL and GeoPandas fail inside Docker — when the native stack breaks a build
• GIS tests pass locally but fail in CI — environment drift, caught by tests
• How to reproject between datums correctly — why grid files change results
• How to record run metadata and data lineage in a GIS pipeline — storing the environment with the output
• What GDAL and OGR actually are — the layer underneath everything

FAQ #

Why is pinning requirements.txt not enough? #

It pins Python packages, but the computation happens in GEOS, PROJ and GDAL. Those are bundled inside wheels or linked from the system, and their versions can differ between installs of identical Python pins.

What is the difference between conda and pip for GIS? #

conda packages the C libraries as first-class packages with their own versions, so a conda lockfile pins GEOS, PROJ and GDAL. pip gets them as opaque payloads inside wheels, or links against whatever the system provides.

Why did my coordinates change by a few millimetres? #

Almost certainly a PROJ transformation grid changed, or one was downloaded on one run and not another. Install proj-data, set PROJ_NETWORK=OFF, and log Transformer.description.

Do I need Docker? #

Not for analysis you run yourself. For anything scheduled, shared, or expected to produce identical output over years, a digest-pinned image is the level that actually holds.

What should I commit? #

The intent file (pixi.toml, environment.yml, requirements.in) and the lockfile. The lockfile is the reproducible artefact; the intent file is what you edit.

How do I know two GDAL versions are loaded? #

Compare rasterio.__gdal_version__ with fiona.__gdal_version__. If they differ, both are present and behaviour depends on link order.

Should I pin exact versions or ranges? #

Ranges in the intent file, exactness in the lockfile. That way upgrading is a deliberate act of regenerating the lock, and the lock is what anyone installs from.