How to Containerise a Python GIS Pipeline with Docker

Problem statement

The pipeline works on your laptop. On the server it does not:

ImportError: libproj.so.25: cannot open shared object file
pyproj.exceptions.CRSError: Invalid projection: EPSG:27700
rasterio: GDAL 3.4 does not support the COG driver

Python GIS packages are thin wrappers over C libraries — GDAL, GEOS, PROJ — and those libraries live outside the virtualenv. requirements.txt pins geopandas==1.0.1; it says nothing about which GDAL is installed, whether PROJ can find its database, or which drivers were compiled in. Two machines with identical pip freeze output can behave differently, and usually do.

A container fixes the layer pip cannot reach. It packages the native libraries, the Python packages, the environment variables and your code into one image that runs identically on your laptop, the server, and CI.

Where the friction usually is:

images that balloon to 3 GB because GDAL was built from source unnecessarily
rebuilding everything on every code change because the layer order is wrong
data baked into the image instead of mounted at run time
files written as root inside the container that nobody can delete outside it
credentials copied into a layer, where they stay forever

Quick answer

Start from a maintained GDAL base image, install Python deps in their own layer, copy code last:

base on ghcr.io/osgeo/gdal:ubuntu-small-* — GDAL, PROJ and GEOS already built
install requirements before copying code, so the dependency layer caches
run as a non-root user with your host UID
mount data at run time; never COPY datasets into the image
pass configuration and secrets as environment variables or mounted files

FROM ghcr.io/osgeo/gdal:ubuntu-small-3.9.2

ENV PYTHONUNBUFFERED=1 \
    PYTHONDONTWRITEBYTECODE=1 \
    PIP_NO_CACHE_DIR=1

RUN apt-get update && apt-get install -y --no-install-recommends \
      python3-pip python3-venv \
 && rm -rf /var/lib/apt/lists/*

WORKDIR /app
COPY requirements.txt .
RUN pip3 install --break-system-packages -r requirements.txt

COPY src/ ./src/
RUN useradd -m -u 1000 gis && chown -R gis /app
USER gis

ENTRYPOINT ["python3", "-m", "src.pipeline"]

docker build -t gis-pipeline:latest .
docker run --rm \
  -v "$PWD/data:/data" \
  -e GIS_DATA_ROOT=/data \
  gis-pipeline:latest --config /data/configs/daily.yml

Copying requirements.txt before src/ is the single most valuable line: a code change then reuses the cached dependency layer, turning a five-minute rebuild into five seconds.

What the image actually contains

Layered anatomy of a GIS container image from OS through GDAL and Python packages to code. — Four layers, and only the top one changes when you edit code.

Step-by-step solution

Vertical steps: choose base, pin dependencies, order layers, add user, mount data, verify. — Six decisions — the first two determine whether the image is 400 MB or 3 GB.

Choose a base image that already has GDAL

Building GDAL from source is slow, fragile and almost never necessary.

Base	Size	Notes
`ghcr.io/osgeo/gdal:ubuntu-small-3.9.2`	~350 MB	Official, common drivers, best default
`ghcr.io/osgeo/gdal:ubuntu-full-3.9.2`	~1.2 GB	Every driver, including proprietary-adjacent ones
`python:3.12-slim` + `pip install geopandas`	~600 MB	Wheels bundle their own GDAL — fine for pure GeoPandas
`mambaorg/micromamba`	~700 MB	Best when you need PyQGIS-adjacent or conda-only packages
`qgis/qgis:release-3_40`	~2.5 GB	Only when you need PyQGIS itself

Pin the tag. :latest means your build is not reproducible, and a GDAL minor release can change driver behaviour.

FROM ghcr.io/osgeo/gdal:ubuntu-small-3.9.2

Check what you actually got:

docker run --rm ghcr.io/osgeo/gdal:ubuntu-small-3.9.2 gdalinfo --version
docker run --rm ghcr.io/osgeo/gdal:ubuntu-small-3.9.2 ogrinfo --formats | head -20

Pin the Python dependencies properly

# requirements.in — what you want
geopandas==1.0.1
pyogrio==0.10.0
rasterio==1.4.3
shapely==2.0.6
pyyaml==6.0.2

pip-compile requirements.in -o requirements.txt --generate-hashes

Hashes make the build tamper-evident and fully reproducible. If a package must match the image's GDAL — rasterio and fiona both link against it — either use the wheels' bundled libraries consistently or install the distro packages, but do not mix the two in one image.

Order the layers so the cache works

Docker caches per instruction and invalidates everything after the first change.

# ── rarely changes ──────────────────────────────────────────────
FROM ghcr.io/osgeo/gdal:ubuntu-small-3.9.2
RUN apt-get update && apt-get install -y --no-install-recommends \
      python3-pip python3-venv \
 && rm -rf /var/lib/apt/lists/*

# ── changes when dependencies change ────────────────────────────
WORKDIR /app
COPY requirements.txt .
RUN pip3 install --break-system-packages --no-cache-dir -r requirements.txt

# ── changes on every commit ─────────────────────────────────────
COPY pyproject.toml ./
COPY src/ ./src/
RUN pip3 install --break-system-packages --no-deps -e .

Add a .dockerignore, or the build context includes your data and your .git folder:

data/
_site/
.git/
.venv/
**/__pycache__/
*.gpkg
*.tif
.env

Run as a real user

Files written by a root container are owned by root on the host, which is a daily annoyance and a security smell.

ARG UID=1000
RUN useradd --create-home --uid ${UID} gis
USER gis

docker build --build-arg UID=$(id -u) -t gis-pipeline .
docker run --rm -u "$(id -u):$(id -g)" -v "$PWD/data:/data" gis-pipeline

Mount data, do not bake it in

docker run --rm \
  -v "$PWD/data:/data:rw" \
  -v "$PWD/configs:/configs:ro" \
  -e GIS_DATA_ROOT=/data \
  gis-pipeline:latest --config /configs/daily.yml

Data in the image makes it enormous, stale the moment it is built, and impossible to share. Mount read-only where you can — :ro on the config volume prevents an accidental write to something version-controlled.

For cloud storage, GDAL's virtual filesystems avoid the mount entirely:

docker run --rm \
  -e AWS_ACCESS_KEY_ID -e AWS_SECRET_ACCESS_KEY -e AWS_REGION \
  -e GDAL_DISABLE_READDIR_ON_OPEN=EMPTY_DIR \
  gis-pipeline:latest --input /vsis3/my-bucket/parcels.gpkg

Set the environment GDAL and PROJ expect

ENV PYTHONUNBUFFERED=1 \
    GDAL_CACHEMAX=512 \
    GDAL_NUM_THREADS=ALL_CPUS \
    CPL_VSIL_CURL_CACHE_SIZE=200000000 \
    VSI_CACHE=TRUE \
    OGR_GEOMETRY_ACCEPT_UNCLOSED_RING=NO

PYTHONUNBUFFERED=1 is not optional: without it, docker logs shows nothing for the first several minutes of a run because stdout is block-buffered.

Verify the image before you trust it

# a build-time smoke test — fails the build, not the 2 a.m. run
RUN python3 -c "\
import geopandas, pyproj, rasterio, shapely; \
from pyproj import CRS; \
assert CRS.from_user_input('EPSG:27700').to_epsg() == 27700; \
print('geopandas', geopandas.__version__, '| GDAL', rasterio.__gdal_version__)"

docker run --rm gis-pipeline:latest python3 -c "
import geopandas as gpd
from shapely.geometry import Point
gdf = gpd.GeoDataFrame(geometry=[Point(-3.19, 55.95)], crs=4326).to_crs(27700)
print(gdf.geometry.iloc[0])"

A PROJ database that cannot resolve EPSG:27700 is the classic broken-image symptom, and this catches it at build time rather than in production.

Code examples

Example 1: a production Dockerfile

# syntax=docker/dockerfile:1
FROM ghcr.io/osgeo/gdal:ubuntu-small-3.9.2 AS base

ENV PYTHONUNBUFFERED=1 \
    PYTHONDONTWRITEBYTECODE=1 \
    PIP_NO_CACHE_DIR=1 \
    PIP_DISABLE_PIP_VERSION_CHECK=1 \
    GDAL_CACHEMAX=512

RUN apt-get update && apt-get install -y --no-install-recommends \
      python3-pip python3-dev build-essential \
 && rm -rf /var/lib/apt/lists/*

WORKDIR /app

FROM base AS deps
COPY requirements.txt .
RUN pip3 install --break-system-packages -r requirements.txt \
 && apt-get purge -y build-essential python3-dev && apt-get autoremove -y

FROM deps AS app
COPY pyproject.toml README.md ./
COPY src/ ./src/
RUN pip3 install --break-system-packages --no-deps -e .

RUN python3 -c "import geopandas, rasterio; from pyproj import CRS; \
    assert CRS.from_epsg(27700).to_epsg() == 27700; print('smoke ok')"

ARG UID=1000
RUN useradd --create-home --uid ${UID} gis && chown -R gis /app
USER gis

HEALTHCHECK --interval=30s --timeout=5s \
  CMD python3 -c "import geopandas" || exit 1

ENTRYPOINT ["python3", "-m", "src.pipeline"]
CMD ["--help"]

Purging the build tools in the same stage that used them keeps them out of the final image.

Example 2: docker compose for a pipeline plus PostGIS

# compose.yaml
services:
  db:
    image: postgis/postgis:16-3.4
    environment:
      POSTGRES_DB: gis
      POSTGRES_USER: gis
      POSTGRES_PASSWORD_FILE: /run/secrets/db_password
    secrets: [db_password]
    volumes:
      - pgdata:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U gis"]
      interval: 5s
      retries: 10

  pipeline:
    build:
      context: .
      args:
        UID: "${UID:-1000}"
    depends_on:
      db: { condition: service_healthy }
    environment:
      GIS_DATA_ROOT: /data
      PGHOST: db
      PGUSER: gis
      PGDATABASE: gis
    secrets: [db_password]
    volumes:
      - ./data:/data
      - ./configs:/configs:ro
    command: ["--config", "/configs/daily.yml"]

secrets:
  db_password:
    file: ./secrets/db_password.txt

volumes:
  pgdata:

UID=$(id -u) docker compose run --rm pipeline

A healthcheck plus depends_on: condition: service_healthy removes the classic "database not ready yet" race on the first run.

Example 3: run the container from cron or systemd

#!/usr/bin/env bash
# /srv/gis/run_container.sh
set -euo pipefail

IMAGE="ghcr.io/acme/gis-pipeline:2026.08.11"
LOG="/srv/gis/logs/pipeline-$(date +%Y%m%d).log"

{
  echo "=== start $(date -Is) image=$IMAGE ==="
  docker run --rm \
    --name gis-pipeline-nightly \
    -u "$(id -u):$(id -g)" \
    -v /srv/gis/data:/data \
    -v /srv/gis/configs:/configs:ro \
    --env-file /srv/gis/job.env \
    --memory 8g --cpus 4 \
    "$IMAGE" --config /configs/daily.yml
  echo "=== done $(date -Is) exit=$? ==="
} >> "$LOG" 2>&1

30 2 * * * /srv/gis/run_container.sh

Pinning an immutable tag means the nightly job runs the image you tested, not whatever :latest points at today. --memory turns a runaway job into a contained failure rather than an unresponsive host.

Example 4: a test that the image is sound

# tests/test_container.py — run against a built image in CI
import json, subprocess

IMAGE = "gis-pipeline:test"

def run(*args):
    return subprocess.run(["docker", "run", "--rm", IMAGE, *args],
                          capture_output=True, text=True, check=True).stdout

def test_gdal_and_proj_agree():
    out = run("python3", "-c",
              "import rasterio, pyproj, json;"
              "print(json.dumps({'gdal': rasterio.__gdal_version__,"
              " 'proj': pyproj.proj_version_str}))")
    versions = json.loads(out)
    assert versions["gdal"].startswith("3.")

def test_can_reproject():
    out = run("python3", "-c",
              "import geopandas as gpd; from shapely.geometry import Point;"
              "g = gpd.GeoDataFrame(geometry=[Point(-3.19, 55.95)], crs=4326).to_crs(27700);"
              "print(round(g.geometry.iloc[0].x))")
    assert 300_000 < int(out.strip()) < 400_000

def test_runs_as_non_root():
    assert run("id", "-u").strip() != "0"

Explanation

A Python GIS environment is really two environments stacked. The upper one is Python packages, which pip and a lock file describe completely. The lower one is native libraries — GDAL with its driver set, GEOS, PROJ with its coordinate-operation database — and nothing in requirements.txt describes it at all. Most "works on my machine" problems in GIS live in that lower layer.

Grid comparing venv, conda and container across what each pins and where it runs. — A venv pins Python packages; a container pins everything below them too.

A container captures both. The image is a filesystem with the native libraries, the Python packages, the environment variables and your code already in place, so "it worked in CI" and "it worked on the server" become the same statement. That is the entire argument, and it is a strong one for pipelines that must run unattended.

The build mechanics reward a little care. Docker caches each instruction and invalidates every layer after the first change, so copying requirements.txt and installing before copying src/ is what keeps a code-only rebuild to seconds. A .dockerignore matters more than people expect: without it, the build context includes data/ and .git/, which can be gigabytes sent to the daemon on every build.

Two operational details cause most of the day-to-day pain. The first is user identity: a container running as root writes root-owned files into your mounted volume, which you then cannot delete. Creating a user with the host's UID fixes it permanently. The second is buffering — Python block-buffers stdout when it is not a terminal, so docker logs stays empty for minutes unless PYTHONUNBUFFERED=1 is set.

Finally, keep data out of images and pin your tags. An image is code and environment; data belongs on a mounted volume or in object storage, where it can change without a rebuild. And a pinned, immutable tag such as 2026.08.11 is what makes a scheduled run reproducible — :latest quietly turns every night into a new experiment.

Edge cases or notes

rasterio/fiona wheels bundle their own GDAL: Installing them on a GDAL base image gives you two GDALs. Either use --no-binary for those packages or accept the wheels' copy and skip the system one.
--break-system-packages on Ubuntu 23.10+: Externally managed environments block pip. Use the flag, or create a venv inside the image.
Alpine is a trap for GIS: musl-based images lack manylinux wheel support for most geospatial packages, so everything compiles from source.
PROJ_LIB / PROJ_DATA moved: PROJ 9 renamed the variable. Prefer a base image that sets it correctly over setting it yourself.
Multi-arch builds: On Apple Silicon, docker build --platform linux/amd64 matters when the server is x86; emulated GDAL is slow.
Layer size: Every RUN creates a layer, so clean apt lists in the same instruction that created them, not in a later one.
Time zone and locale: Containers default to UTC and the C locale, which can change date parsing and string sorting compared with your laptop.

Internal links

FAQ

Which base image should I use for a Python GIS pipeline?

ghcr.io/osgeo/gdal:ubuntu-small-<version> for most work — GDAL, GEOS and PROJ are already built and it stays around 350 MB. Use python:3.12-slim plus wheels if you only need GeoPandas, and the QGIS image only when you need PyQGIS.

Why is my image several gigabytes?

Usually a full GDAL base you do not need, build tools left in the final image, or data copied in. Use the -small base, purge build dependencies in the same layer, and add a .dockerignore.

Why does every build reinstall all the dependencies?

Because code is copied before requirements.txt. Copy the requirements file and install first, then copy the source, so the dependency layer stays cached.

How do I stop the container writing root-owned files?

Create a user with the host UID (useradd -u 1000) and USER it in the Dockerfile, or pass -u "$(id -u):$(id -g)" at run time.

Should data go inside the image?

No. Mount it as a volume or read it from object storage with GDAL's /vsis3/, /vsigs/ handlers. Baking data in bloats the image and makes it stale immediately.

Why do I see no logs until the run finishes?

Python buffers stdout when it is not a terminal. Set ENV PYTHONUNBUFFERED=1 in the Dockerfile, or run with python3 -u.

How do I keep the container's GDAL and Python packages consistent?

Pin both: an exact base image tag and a hash-pinned requirements.txt. Avoid mixing wheel-bundled GDAL with the system GDAL, and add a build-time smoke test that reprojects a point.

How to Containerise a Python GIS Pipeline with Docker #

Problem statement #

Quick answer #

What the image actually contains #

Step-by-step solution #

Choose a base image that already has GDAL #

Pin the Python dependencies properly #

Order the layers so the cache works #

Run as a real user #

Mount data, do not bake it in #

Set the environment GDAL and PROJ expect #

Verify the image before you trust it #

Code examples #

Example 1: a production Dockerfile #

Example 2: docker compose for a pipeline plus PostGIS #

Example 3: run the container from cron or systemd #

Example 4: a test that the image is sound #

Explanation #

Edge cases or notes #

Internal links #

FAQ #

Which base image should I use for a Python GIS pipeline? #

Why is my image several gigabytes? #

Why does every build reinstall all the dependencies? #

How do I stop the container writing root-owned files? #

Should data go inside the image? #

Why do I see no logs until the run finishes? #

How do I keep the container's GDAL and Python packages consistent? #