GIS Tests Pass Locally but Fail in CI: How to Fix It #

Problem statement #

The suite is green on your laptop. You push, and CI reports failures in tests that have not been touched for weeks.

FAILED tests/test_transforms.py::test_buffer_area
    assert 7853.98 == approx(7853.9816, rel=1e-09)

FAILED tests/test_clean.py::test_fixes_invalid_geometry
    AssertionError: 1 invalid geometries remain

FAILED tests/test_io.py::test_reproject_to_bng
    pyproj.exceptions.CRSError: Invalid projection: EPSG:27700

Three different errors, one cause: the machine running the tests is not the machine you wrote them on. Python GIS has an unusually deep native stack — GEOS, PROJ, GDAL — and every layer of it can differ between environments while pip list looks identical.

The tests are not flaky. They are correctly reporting that your code depends on something you never pinned.

Quick answer #

Print the native stack in CI before anything else runs, then pin it:

# .github/workflows/ci.yml
- name: Report the native stack
  run: |
    python -c "
    import geopandas, shapely, pyproj, rasterio, fiona
    print('geopandas', geopandas.__version__)
    print('shapely  ', shapely.__version__, '/ GEOS', shapely.geos_version_string)
    print('pyproj   ', pyproj.__version__, '/ PROJ', pyproj.proj_version_str)
    print('rasterio ', rasterio.__version__, '/ GDAL', rasterio.__gdal_version__)
    print('fiona    ', fiona.__version__)
    print('PROJ data', pyproj.datadir.get_data_dir())
    "

Compare that block with the same command locally. The line that differs is your bug. Then fix it at the level it belongs to:

Where the two environments diverge #

Step-by-step solution #

1. Make the environment visible before you debug it #

You cannot diagnose drift you cannot see. Add the version dump as the first step of every CI job and keep it there permanently — it costs two seconds and turns the next occurrence into a thirty-second diagnosis.

- name: Environment
  run: |
    python --version
    python -c "import shapely, pyproj, rasterio; print(shapely.geos_version_string, pyproj.proj_version_str, rasterio.__gdal_version__)"
    python -c "import fiona; print(sorted(fiona.supported_drivers))"

The driver list matters more than people expect. A GDAL built without GPKG write support fails only in the test that writes a GeoPackage, with a message about an unsupported driver that reads like a code bug.

2. Fix numeric drift by loosening the assertion, not the code #

Areas, lengths and buffer vertices legitimately differ between GEOS releases. A buffer is an approximation of a circle, and how many segments it uses per quarter is an implementation detail.

# fails whenever GEOS changes
assert out.geometry.area.iloc[0] == pytest.approx(7853.9816, rel=1e-9)

# survives GEOS upgrades, still catches a metres/degrees mistake
assert out.geometry.area.iloc[0] == pytest.approx(math.pi * 50 ** 2, rel=1e-3)

rel=1e-3 still fails by a factor of 10^10 if someone buffers in degrees, which is the bug you were testing for. Precision beyond that is testing GEOS, not your pipeline.

3. Fix CRSError by shipping the PROJ database #

EPSG:27700 is not built into pyproj — it is looked up in proj.db, a file that ships with the PROJ installation. Wheels bundle it; some minimal containers and some conda combinations do not.

# diagnose
import pyproj
print(pyproj.datadir.get_data_dir())        # where it is looking
print(pyproj.database.get_units_map() and "proj.db readable")

If the directory is wrong or empty:

# Docker: install PROJ data explicitly, do not rely on inheritance
RUN apt-get update && apt-get install -y --no-install-recommends proj-data proj-bin \
    && rm -rf /var/lib/apt/lists/*
ENV PROJ_DATA=/usr/share/proj

# GitHub Actions: pyproj's own wheels carry proj.db — prefer them
- run: pip install --only-binary=:all: pyproj shapely geopandas

Mixing a conda pyproj with a pip rasterio is the reliable way to end up with two PROJ installations and a PROJ_DATA pointing at the wrong one.

4. Fix validity differences by asserting on the outcome #

GEOS changed how make_valid repairs certain self-intersections. A test that asserts the exact repaired geometry will break on the upgrade; a test that asserts the repair worked will not.

# brittle
assert repaired.geometry.iloc[0].wkt == "POLYGON ((0 0, 1 0, 1 1, 0 1, 0 0))"

# durable
assert repaired.is_valid.all()
assert repaired.geometry.area.sum() == pytest.approx(original_area, rel=0.01)
assert (repaired.geom_type.isin(["Polygon", "MultiPolygon"])).all()

5. Fix "passes alone, fails in the suite" — that one is yours #

If a test passes with pytest tests/test_clean.py::test_x and fails in a full run, the environment is innocent. Something is shared:

# the usual culprits
def test_writes_output():
    gdf.to_file("output.gpkg")            # relative path — depends on CWD
    ...

@pytest.fixture(scope="session")          # session scope + mutation = order dependence
def parcels():
    return gpd.read_file(FIXTURE)

Fixes: use tmp_path for every write, make mutable fixtures function-scoped, and confirm with pytest -p no:randomly versus pytest --randomly-seed=last.

def test_writes_output(tmp_path, parcels):
    out = tmp_path / "output.gpkg"        # unique per test, cleaned up
    parcels.to_file(out)
    assert gpd.read_file(out).crs == parcels.crs

6. Stop the drift returning: pin the whole stack #

Pinning Python packages is not enough, because the native libraries come with the wheels or from the OS. Pick one strategy and apply it everywhere.

# Strategy A — a container, identical everywhere
FROM ghcr.io/osgeo/gdal:ubuntu-small-3.8.4
RUN pip install --no-cache-dir -r requirements.lock

# Strategy B — conda-lock, resolved once, reused by every machine
- uses: conda-incubator/setup-miniconda@v3
  with:
    environment-file: conda-lock.yml

# Strategy C — pip with hashes and binary-only, simplest and usually enough
geopandas==1.0.1 --hash=sha256:...
shapely==2.0.6 --hash=sha256:...
pyproj==3.6.1 --hash=sha256:...

Strategy C plus --only-binary=:all: gets most teams to reproducible builds, because the wheels bundle their own GEOS, PROJ and GDAL. It stops working the moment you need a GDAL driver the wheels omit — then it is A.

Code examples #

Example 1: a test that pins the stack itself #

If a specific version matters, assert it. A clear failure at the top of the run beats a confusing one in the middle.

# tests/test_environment.py
import pyproj, shapely, pytest

MIN_GEOS = (3, 11, 0)

def test_geos_is_recent_enough():
    assert shapely.geos_version >= MIN_GEOS, (
        f"GEOS {shapely.geos_version} is older than {MIN_GEOS}; "
        "make_valid behaves differently below this"
    )

def test_proj_database_is_present():
    # this raises CRSError long before any pipeline code runs
    assert pyproj.CRS.from_epsg(27700).to_epsg() == 27700

@pytest.mark.parametrize("driver", ["GPKG", "GeoJSON", "ESRI Shapefile"])
def test_required_drivers_available(driver):
    import fiona
    assert driver in fiona.supported_drivers
    assert "w" in fiona.supported_drivers[driver]

Example 2: a CI matrix that surfaces drift instead of hiding it #

jobs:
  test:
    strategy:
      fail-fast: false
      matrix:
        include:
          - { os: ubuntu-latest,  python: "3.11" }
          - { os: ubuntu-latest,  python: "3.12" }
          - { os: macos-latest,   python: "3.12" }
    runs-on: ${{ matrix.os }}
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python }}
          cache: pip
      - run: pip install --only-binary=:all: -r requirements.txt
      - name: Environment
        run: python -c "import shapely,pyproj,rasterio;print(shapely.geos_version_string,pyproj.proj_version_str,rasterio.__gdal_version__)"
      - run: pytest -q

fail-fast: false is the important line. Without it, the first failing cell cancels the others and you never learn whether the problem is Python 3.12 or macOS.

Example 3: reproducing CI locally before you push #

# same image CI uses, same lock file, your working tree
docker run --rm -it \
  -v "$PWD":/work -w /work \
  ghcr.io/osgeo/gdal:ubuntu-small-3.8.4 \
  bash -c "pip install -q -r requirements.lock && pytest -q"

Two minutes here beats eight push-and-wait cycles.

Explanation #

pip install geopandas installs a Python package that is mostly a wrapper. The work happens in C and C++ libraries: GEOS does geometry operations, PROJ does coordinate transformation, GDAL/OGR does file and format handling. Those libraries have their own versions, their own release cadence, and their own bugs.

Wheels bundle a copy of each. Conda installs them as separate packages. A system Python may use whatever the distribution ships. So two environments can have identical pip freeze output and different GEOS versions — and GEOS is exactly what decides whether your buffer has 8 or 16 segments per quarter circle, and how a bowtie polygon gets repaired.

This is why "works on my machine" is more common in GIS than in most Python work, and why the fix is almost never in the test file. The test was right. It found a real difference between two environments that you had been assuming were the same.

The corollary is worth stating plainly: a test that fails in CI has done its job. The instinct to loosen the assertion until it passes is right when the assertion was testing GEOS, and wrong when it was testing your code. Step 2 above is the first case. Step 5 is the second.

Edge cases or notes #

• Locale changes number parsing. A CI runner with a comma decimal separator can break CSV reads that work locally. Set LC_ALL=C in CI.
• Timezone affects date columns. Runners default to UTC; laptops do not. Store and assert on timezone-aware datetimes, or fix TZ in the workflow.
• Memory limits are lower in CI than on your laptop. A test that passes locally and gets killed with exit code 137 in CI is an OOM, not a logic failure — see batch GIS jobs that run out of memory.
• fiona.supported_drivers differs between GDAL builds. KML, FileGDB and some raster drivers are frequently absent from minimal builds.
• Shapely 1.x and 2.x differ materially. shapely.geos_version is the check; the 2.x rewrite changed array behaviour, not just the API.
• Caching pip in CI can preserve a broken resolution. When debugging drift, clear the cache once before concluding anything.
• Do not pip install --upgrade in CI. It makes every run a different environment, which is the problem you are trying to solve.

Internal links #

• How to test a GIS pipeline with pytest — writing the suite this page debugs
• What to test in a GIS pipeline — which assertions survive an environment change
• How to run a Python GIS pipeline in CI with GitHub Actions — the workflow these fixes go into
• How to containerise a Python GIS pipeline with Docker — the strongest answer to drift
• How to make a GIS workflow reproducible in Python — pinning as a general practice
• ModuleNotFoundError in a scheduled GIS job — the same class of problem under cron
• Fiona vs pyogrio: how GeoPandas reads and writes files — which engine is actually doing your I/O

FAQ #

Why do buffer areas differ by a tiny amount between machines? #

A buffer approximates a circle with straight segments. The default number of segments per quarter circle is a GEOS implementation detail and has changed between releases. Use a relative tolerance of 1e-3 or looser.

My CI says CRSError: Invalid projection: EPSG:27700 but the code works locally. Why? #

PROJ cannot find proj.db. Print pyproj.datadir.get_data_dir() in CI — it will point somewhere empty or wrong. Install proj-data, set PROJ_DATA, or install pyproj from wheels which bundle it.

Should I pin GEOS and PROJ versions directly? #

You cannot pin them from requirements.txt — they arrive inside wheels or from the OS. Pin them by fixing the whole environment: a container image, a conda lock file, or hash-pinned binary-only wheels.

Is it acceptable to skip a test in CI? #

Only for genuine environment limitations — a driver that is not available, a database that is not running — and always with pytest.mark.skipif and a reason string. Skipping to make the build green hides the drift for the next person.

Why does my test suite pass individually but fail together? #

Shared state: a relative output path, a session-scoped fixture that a test mutates, or a global set by one test. This is a bug in the tests, not the environment. Use tmp_path and function-scoped fixtures.

How do I know whether to fix the code or the assertion? #

Ask what the assertion is protecting against. If loosening it would still catch a metres/degrees mix-up or a dropped CRS, loosen it. If loosening it makes the test pass on incorrect output, fix the environment instead.

Does using Docker in CI solve all of this? #

It solves version drift completely, because the image is the environment. It does not solve locale, timezone, memory limits or test interdependence — those still need the fixes above.