How to Build a Small, Fast Python GIS Docker Image

Problem statement

The obvious Dockerfile produces something large and slow to work with:

FROM python:3.12
COPY . /app
RUN pip install -r /app/requirements.txt

Built for this page around a 12-line clipping script and two requirements, geopandas and rasterio, it measured 1.66 GB (Example 2). And because COPY . /app sits above the install, every code change reinstalls every package: 12 s for a one-line edit, against a fifth of a second once the layers are ordered.

Try to shrink it with Alpine and it stops working entirely:

FROM python:3.12-alpine
RUN pip install -r requirements.txt

Downloading rasterio-1.5.1.tar.gz (457 kB)
ERROR: A GDAL API version must be specified. Provide a path to gdal-config using a GDAL_CONFIG
environment variable or use a GDAL_VERSION environment variable.

And the slim image that looks like the answer builds cleanly and is broken: import rasterio fails with libexpat.so.1: cannot open shared object file.

All three have the same root cause: Python GIS is a thin layer over large C libraries, and how you obtain those libraries decides everything.

Quick answer

For the smallest working image, start from python:3.12-slim, add the one system library rasterio's wheel expects, and order the layers so a code change rebuilds only the code:

FROM python:3.12-slim

ENV PIP_NO_CACHE_DIR=1 PIP_DISABLE_PIP_VERSION_CHECK=1 PYTHONDONTWRITEBYTECODE=1
RUN apt-get update && apt-get install -y --no-install-recommends libexpat1 \
    && rm -rf /var/lib/apt/lists/*

WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY src/ ./src/

ENTRYPOINT ["python", "-m", "src.pipeline"]

Measured: 538 MB, with GDAL 3.12.4 inside the wheels. If you need one GDAL shared by everything — a newer one, or a driver the wheels leave out — use a GDAL base image and compile the bindings against it in a multi-stage build (Example 1): 668 MB and a single GDAL 3.13.3.

1,333 NUTS 3 regions in 36 countries; share aged 65+ in 2023: 2.7–36.0%
{'intermediate': 619, 'rural': 438, 'urban': 276} 1,332 distinct names

Base image, stages, layer order, non-root

1. Choose the base image deliberately

Built for this page from the same two requirements (Example 2):

Alpine is the trap. It uses musl libc, and PyPI's manylinux wheels are built for glibc. Some GIS packages now publish musllinux wheels as well; rasterio does not, so pip downloads its source and stops at gdal-config.

Slim works, with one line. shapely, pyproj, pyogrio and rasterio publish manylinux wheels with their native libraries bundled — but rasterio's expects the system's libexpat.so.1, which python:3.12-slim does not include. The build succeeds; the import fails at run time. apt-get install libexpat1 costs about a megabyte.

A GDAL base does not share its GDAL by default. pip install rasterio on top of it still installs the wheel, and the wheel loads its own bundled GDAL: measured, the base's GDAL was 3.13.3 while rasterio and pyogrio both reported 3.12.4. The base image's copy sits unused. It only becomes the shared stack when the bindings are compiled against it with --no-binary, which needs a compiler — the job multi-stage builds exist for.

2. Use a multi-stage build when you compile

The build needs compilers, headers and pip's machinery. The runtime needs none of it:

FROM ghcr.io/osgeo/gdal:ubuntu-small-3.13.3 AS build
RUN apt-get update && apt-get install -y --no-install-recommends \
        python3-pip python3-venv build-essential python3-dev \
    && rm -rf /var/lib/apt/lists/*
RUN python3 -m venv /venv
ENV PATH="/venv/bin:$PATH"
COPY requirements.txt .
RUN pip install --no-binary rasterio,pyogrio -r requirements.txt

FROM ghcr.io/osgeo/gdal:ubuntu-small-3.13.3
RUN apt-get update && apt-get install -y --no-install-recommends python3 \
    && rm -rf /var/lib/apt/lists/*
COPY --from=build /venv /venv
ENV PATH="/venv/bin:$PATH"

Installing into a virtualenv is what makes the copy clean: /venv is one self-contained directory holding every installed package, so COPY --from=build /venv /venv moves the whole environment in one instruction.

What it saves depends on what the build stage held. With wheels only, splitting the stages took the GDAL-base image from 896 MB to 836 MB; compiling the bindings instead, and so dropping the wheels' own GDAL, took it to 668 MB — grid included.

3. Order layers so a code change rebuilds one layer

Docker caches per instruction and invalidates everything after the first change. So the order is: rarely-changing first, frequently-changing last.

# ❌ any source change reinstalls every dependency
COPY . /app
RUN pip install -r /app/requirements.txt

# ✅ dependencies are cached until requirements.txt changes
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY src/ ./src/

Measured after a one-line change to the script: 12 s to rebuild the naive image, about 0.2 s for every ordered one (Example 2). Cold, the wheel-based builds took 11 to 22 s here — pip installs wheels quickly — while compiling the bindings took 118 s (Example 1), which is the build this ordering saves you from repeating.

Add a .dockerignore, or the build context includes everything:

.git
.venv
__pycache__/
*.pyc
data/
notebooks/
.pytest_cache
*.egg-info

Without it, COPY . /app sends your entire data/ directory to the daemon on every build, and a change to any file in it invalidates the copy.

4. Strip what you do not need

ENV PIP_NO_CACHE_DIR=1 \
    PIP_DISABLE_PIP_VERSION_CHECK=1 \
    PYTHONDONTWRITEBYTECODE=1

RUN apt-get update \
    && apt-get install -y --no-install-recommends python3 \
    && rm -rf /var/lib/apt/lists/*

• --no-install-recommends stops apt pulling in documentation, locales and suggested packages.
• rm -rf /var/lib/apt/lists/* in the same RUN — a separate RUN would not shrink the image, because the earlier layer still contains the files.
• PIP_NO_CACHE_DIR=1 keeps pip's wheel cache out of the layer.
• PYTHONDONTWRITEBYTECODE=1 skips .pyc files, which will be regenerated at runtime anyway.

What the first and third are worth for this stack was measured in Example 2 rather than guessed: keeping pip's cache made the slim image 123 MB larger (660 MB against 537 MB), while --no-install-recommends made no difference to the GDAL-base image, whose two packages recommend nothing it lacked. Check where the size actually is before optimising further — docker history lists every layer with its size, and Example 2 prints it for the final image.

5. Keep the native stack deterministic

An image is only worth building if it behaves the same every time:

FROM ghcr.io/osgeo/gdal:ubuntu-small-3.13.3@sha256:64250faf833c06d4b21afce4c27190039ba7ab58d70f0eebc87cf77d929c0b40

ENV PROJ_NETWORK=OFF
RUN pyproj sync --file uk_os_OSTN15_NTv2_OSGBtoETRS.tif \
        --target-directory "$(python -c 'import pyproj; print(pyproj.datadir.get_data_dir())')"

COPY requirements.txt .
RUN pip install --no-cache-dir --require-hashes -r requirements.txt

• The digest pins the base image byte for byte. A tag such as 3.13.3 can be re-pushed; a sha256: cannot.
• The grid goes where pyproj looks. The pyproj wheel carries its own PROJ and its own data directory, and does not read the base image's /usr/share/proj, so an apt-get install proj-data would not reach it. pyproj sync puts a named grid into pyproj's directory at build time. Measured, Manchester in British National Grid came out 1.8 m apart with and without OSTN15.
• PROJ_NETWORK=OFF makes a missing grid fall back to a less accurate transformation instead of downloading one at run time — which is why the grid has to be in the image.
• --require-hashes makes pip refuse any package whose content differs from a hash-pinned lockfile.

6. Run as a non-root user

RUN useradd --create-home --uid 10001 app
WORKDIR /app
COPY --chown=app:app src/ ./src/
USER app

Containers run as root by default, which writes root-owned files into mounted volumes and gives a compromised process more privilege than it needs. A fixed numeric uid is what volume permissions and Kubernetes runAsUser policies reference.

The catch is the other direction. Measured, the image's uid 10001 could not write into a host directory owned by the host user — sqlite3_open(/out/mainland.gpkg) failed: unable to open database file — while the same run with --user "$(id -u):$(id -g)" wrote its output. Decide which uid owns the output directory before the job runs at 2 a.m.

--chown on the COPY avoids a separate RUN chown, which would duplicate every copied file in a new layer.

A production Dockerfile, measured and verified

The data used on this page

The "districts" are Europe's NUTS 3 regions — 1,333 of them once joined to data — from Eurostat GISCO at 1:1 million, with GISCO's urban–rural typology as the category, and the share of the population aged 65 and over for 2019 to 2023 from the Eurostat table demo_r_pjanind3 as rate_2019 … rate_2023. Eurostat refuses the default python-requests user agent, hence the header.

import math
import os

import geopandas as gpd
import pandas as pd
import requests

YEARS = [2019, 2020, 2021, 2022, 2023]


def eurostat(table, **filters):
    """A Eurostat table as a Series indexed by every dimension."""
    stat = requests.get(f"https://ec.europa.eu/eurostat/api/dissemination/statistics/1.0/data/{table}",
                        params={"format": "JSON", **filters},
                        headers={"User-Agent": "spatialworkflow.io guide"}, timeout=120).json()
    codes = [sorted(stat["dimension"][d]["category"]["index"], key=stat["dimension"][d]["category"]["index"].get)
             for d in stat["id"]]
    values = [stat["value"].get(str(i)) for i in range(math.prod(stat["size"]))]
    return pd.Series(values, index=pd.MultiIndex.from_product(codes, names=stat["id"]), dtype=float)


if not os.path.exists("districts.gpkg"):
    nuts = gpd.read_file("https://gisco-services.ec.europa.eu/distribution/v2/nuts/geojson/"
                         "NUTS_RG_01M_2021_4326_LEVL_3.geojson")
    older = (eurostat("demo_r_pjanind3", indic_de="PC_Y65_MAX", unit="PC",
                      sinceTimePeriod=YEARS[0], untilTimePeriod=YEARS[-1])
             .droplevel(["freq", "indic_de", "unit"]).unstack("time"))
    older.columns = [f"rate_{year}" for year in older.columns]
    kind = {1: "urban", 2: "intermediate", 3: "rural"}
    (nuts.assign(category=nuts["URBN_TYPE"].map(kind))
         .rename(columns={"NUTS_ID": "code", "CNTR_CODE": "country", "NAME_LATN": "name"})
         [["code", "country", "name", "category", "geometry"]]
         .merge(older, left_on="code", right_index=True)
         .dropna(subset=[f"rate_{YEARS[-1]}"])
         .to_file("districts.gpkg"))

districts = gpd.read_file("districts.gpkg")
print(f"{len(districts):,} NUTS 3 regions in {districts.country.nunique()} countries; "
      f"share aged 65+ in {YEARS[-1]}: {districts[f'rate_{YEARS[-1]}'].min():.1f}–"
      f"{districts[f'rate_{YEARS[-1]}'].max():.1f}%")
print(districts.category.value_counts().to_dict(),
      f"{districts.name.nunique():,} distinct names")

# the pipeline clips Portugal's regions; the same file is the test fixture
for folder in ("src", "tests", "data", "out"):
    os.makedirs(folder, exist_ok=True)
portugal = districts[districts["country"] == "PT"]
portugal.to_file("data/portugal.gpkg")
portugal.to_file("tests/portugal.gpkg")

1,333 NUTS 3 regions in 36 countries; share aged 65+ in 2023: 2.7–36.0%
{'intermediate': 619, 'rural': 438, 'urban': 276} 1,332 distinct names

The pipeline under test clips Portugal's regions to the mainland and writes them in the national grid:

%%writefile src/pipeline.py
"""Clip Portugal's NUTS 3 regions to the mainland and write them in PT-TM06."""
import sys

import geopandas as gpd
from shapely.geometry import box


def main(source="/data/portugal.gpkg", target="/out/mainland.gpkg"):
    regions = gpd.read_file(source)
    mainland = regions.clip(box(-9.6, 36.9, -6.1, 42.2)).to_crs(3763)
    mainland.to_file(target)
    print(f"{len(regions)} regions in, {len(mainland)} on the mainland, "
          f"{mainland.area.sum() / 1e6:,.0f} km² written to {target}")


if __name__ == "__main__":
    main(*sys.argv[1:])

%%writefile src/__init__.py
# makes src/ a package, so `python -m src.pipeline` works

%%writefile requirements.txt
geopandas==1.1.4
rasterio==1.5.1

%%writefile .dockerignore
.git
__pycache__/
*.pyc
data/
out/

Example 1: a complete production Dockerfile

%%writefile Dockerfile
# syntax=docker/dockerfile:1.7
ARG GDAL_IMAGE=ghcr.io/osgeo/gdal:ubuntu-small-3.13.3@sha256:64250faf833c06d4b21afce4c27190039ba7ab58d70f0eebc87cf77d929c0b40

# ─── build ──────────────────────────────────────────────────────────────────
FROM ${GDAL_IMAGE} AS build

ENV PIP_DISABLE_PIP_VERSION_CHECK=1 \
    PYTHONDONTWRITEBYTECODE=1 \
    DEBIAN_FRONTEND=noninteractive

RUN apt-get update && apt-get install -y --no-install-recommends \
        python3-pip python3-venv python3-dev build-essential \
    && rm -rf /var/lib/apt/lists/*

RUN python3 -m venv /venv
ENV PATH="/venv/bin:$PATH"

COPY requirements.txt .
# the GDAL bindings compiled against the base image's GDAL, not wheels with their own
RUN --mount=type=cache,target=/root/.cache/pip \
    pip install --no-binary rasterio,pyogrio -r requirements.txt

# the OSTN15 grid into pyproj's own data directory, so PROJ_NETWORK can stay off
RUN pyproj sync --file uk_os_OSTN15_NTv2_OSGBtoETRS.tif \
        --target-directory "$(python -c 'import pyproj; print(pyproj.datadir.get_data_dir())')"

# ─── test (optional stage, not in the final image) ──────────────────────────
FROM build AS test
WORKDIR /app
COPY requirements-dev.txt .
RUN pip install -r requirements-dev.txt
COPY src/ ./src/
COPY tests/ ./tests/
# python -m puts /app on sys.path; a bare `pytest` would not find src/
RUN python -m pytest -q tests/test_pipeline.py

# ─── runtime ────────────────────────────────────────────────────────────────
FROM ${GDAL_IMAGE} AS runtime

ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get install -y --no-install-recommends python3 \
    && rm -rf /var/lib/apt/lists/* \
    && useradd --create-home --uid 10001 --shell /usr/sbin/nologin app

COPY --from=build /venv /venv

ENV PATH="/venv/bin:$PATH" \
    PROJ_NETWORK=OFF \
    PYTHONUNBUFFERED=1 \
    PYTHONDONTWRITEBYTECODE=1 \
    GDAL_DISABLE_READDIR_ON_OPEN=EMPTY_DIR

WORKDIR /app
COPY --chown=app:app src/ ./src/
USER app

ENTRYPOINT ["python", "-m", "src.pipeline"]

%%writefile requirements-dev.txt
pytest==9.1.1

%%writefile tests/test_pipeline.py
import geopandas as gpd

from src.pipeline import main


def test_clips_to_the_mainland(tmp_path):
    main("tests/portugal.gpkg", str(tmp_path / "mainland.gpkg"))
    out = gpd.read_file(tmp_path / "mainland.gpkg")
    assert out.crs.to_epsg() == 3763
    assert len(out) == 23          # 25 regions less the Azores and Madeira

import re
import subprocess
import time


def sh(*args):
    return subprocess.run(args, capture_output=True, text=True, check=True).stdout


started = time.perf_counter()
test = subprocess.run(["docker", "build", "--no-cache", "--progress=plain", "--target", "test",
                       "-t", "sw-gis:test", "."], capture_output=True, text=True)
summary = re.search(r"\d+ passed in [\d.]+s", test.stderr).group()
print(f"test stage built from nothing in {time.perf_counter() - started:.0f} s: {summary}")
sh("docker", "build", "-q", "-t", "sw-gis:final", ".")
print("runtime image", sh("docker", "images", "sw-gis:final", "--format", "{{.Size}}").strip())
print(sh("docker", "run", "--rm", "--entrypoint", "sh", "sw-gis:final", "-c",
         "gdalinfo --version; python -c \"import rasterio, pyogrio; "
         "print('rasterio', rasterio.__gdal_version__, 'pyogrio', pyogrio.__gdal_version_string__)\"").strip())

test stage built from nothing in 118 s: 1 passed in 0.23s
runtime image 668MB
GDAL 3.13.3 "Iowa City", released 2026/08/13
rasterio 3.13.3 pyogrio 3.13.3

The test stage runs the suite during the build and is not part of the runtime image, so a failing test fails the build; --target test builds only up to it. It runs python -m pytest rather than pytest: a bare pytest does not put the working directory on sys.path, and the first version of this stage failed with No module named 'src'.

--mount=type=cache,target=/root/.cache/pip keeps pip's downloads between builds without their entering a layer. It only works if pip is allowed a cache: PIP_NO_CACHE_DIR=1 in the build stage, which the earlier version of this Dockerfile also set, would switch the mount off.

GDAL_DISABLE_READDIR_ON_OPEN=EMPTY_DIR stops GDAL listing an entire directory when opening one file, which is slow on network storage and worse on object storage.

Example 2: measuring the difference

Every variant from this page, built from nothing, then rebuilt after a one-line change to the script:

variants = {
    "naive": """FROM python:3.12
COPY . /app
RUN pip install -r /app/requirements.txt
WORKDIR /app
""",
    "slim, pip cache kept": """FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY src/ ./src/
""",
    "slim": """FROM python:3.12-slim
ENV PIP_NO_CACHE_DIR=1 PIP_DISABLE_PIP_VERSION_CHECK=1 PYTHONDONTWRITEBYTECODE=1
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY src/ ./src/
""",
    "slim + libexpat1": """FROM python:3.12-slim
ENV PIP_NO_CACHE_DIR=1 PIP_DISABLE_PIP_VERSION_CHECK=1 PYTHONDONTWRITEBYTECODE=1
RUN apt-get update && apt-get install -y --no-install-recommends libexpat1 \\
    && rm -rf /var/lib/apt/lists/*
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY src/ ./src/
""",
    "alpine": """FROM python:3.12-alpine
ENV PIP_NO_CACHE_DIR=1
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY src/ ./src/
""",
    "gdal base, with recommends": """FROM ghcr.io/osgeo/gdal:ubuntu-small-3.13.3
ENV PIP_NO_CACHE_DIR=1 PYTHONDONTWRITEBYTECODE=1 DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get install -y python3-pip python3-venv && rm -rf /var/lib/apt/lists/*
RUN python3 -m venv /venv
ENV PATH="/venv/bin:$PATH"
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY src/ ./src/
""",
    "gdal base": """FROM ghcr.io/osgeo/gdal:ubuntu-small-3.13.3
ENV PIP_NO_CACHE_DIR=1 PYTHONDONTWRITEBYTECODE=1 DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get install -y --no-install-recommends python3-pip python3-venv \\
    && rm -rf /var/lib/apt/lists/*
RUN python3 -m venv /venv
ENV PATH="/venv/bin:$PATH"
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY src/ ./src/
""",
    "gdal, multi-stage": """FROM ghcr.io/osgeo/gdal:ubuntu-small-3.13.3 AS build
ENV PIP_NO_CACHE_DIR=1 PYTHONDONTWRITEBYTECODE=1 DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get install -y --no-install-recommends python3-pip python3-venv \\
    && rm -rf /var/lib/apt/lists/*
RUN python3 -m venv /venv
ENV PATH="/venv/bin:$PATH"
COPY requirements.txt .
RUN pip install -r requirements.txt

FROM ghcr.io/osgeo/gdal:ubuntu-small-3.13.3
RUN apt-get update && apt-get install -y --no-install-recommends python3 \\
    && rm -rf /var/lib/apt/lists/*
COPY --from=build /venv /venv
ENV PATH="/venv/bin:$PATH"
WORKDIR /app
COPY src/ ./src/
""",
}

check = ("import rasterio, pyogrio; "
         "print(f'rasterio GDAL {rasterio.__gdal_version__}, pyogrio GDAL {pyogrio.__gdal_version_string__}')")
print(f"{'variant':28} {'size':>8} {'cold':>6} {'edit':>6}  imports")
for name, dockerfile in variants.items():
    tag = "sw-gis:" + name.replace(" ", "-").replace(",", "").replace("+", "and")
    with open("Dockerfile.variant", "w") as out:
        out.write(dockerfile)
    started = time.perf_counter()
    built = subprocess.run(["docker", "build", "--no-cache", "--progress=plain", "-f", "Dockerfile.variant",
                            "-t", tag, "."],
                           capture_output=True, text=True)
    cold = time.perf_counter() - started
    if built.returncode:
        reason = next((l.split("ERROR: ", 1)[1] for l in built.stderr.splitlines() if "GDAL API version" in l),
                      built.stderr.strip().splitlines()[-1])
        print(f"{name:28} {'—':>8} {cold:5.0f}s {'':>6}  build failed: {reason[:70]}")
        continue
    with open("src/pipeline.py", "a") as out:
        out.write("# an edit\n")
    started = time.perf_counter()
    sh("docker", "build", "-q", "-f", "Dockerfile.variant", "-t", tag, ".")
    edit = time.perf_counter() - started
    source = open("src/pipeline.py").read().splitlines(keepends=True)[:-1]
    open("src/pipeline.py", "w").writelines(source)
    size = sh("docker", "images", tag, "--format", "{{.Size}}").strip()
    ran = subprocess.run(["docker", "run", "--rm", tag, "python", "-c", check], capture_output=True, text=True)
    imports = ran.stdout.strip() or ran.stderr.strip().splitlines()[-1]
    print(f"{name:28} {size:>8} {cold:5.0f}s {edit:5.2f}s  {imports[:60]}")
    sh("docker", "image", "rm", tag)

print("\nfinal image, largest layers:")
for line in sh("docker", "history", "sw-gis:final", "--format", "{{.Size}}\t{{.CreatedBy}}").splitlines():
    size, step = line.split("\t", 1)
    if size.endswith(("MB", "GB")):
        print(f"  {size:>7}  {step[:70]}")

variant                          size   cold   edit  imports
naive                          1.66GB    12s 12.36s  rasterio GDAL 3.12.4, pyogrio GDAL 3.12.4
slim, pip cache kept            660MB    11s  0.19s  ImportError: libexpat.so.1: cannot open shared object file: 
slim                            537MB    11s  0.18s  ImportError: libexpat.so.1: cannot open shared object file: 
slim + libexpat1                538MB    14s  0.17s  rasterio GDAL 3.12.4, pyogrio GDAL 3.12.4
alpine                              —     8s         build failed: A GDAL API version must be specified. Provide a path to gdal-config us
gdal base, with recommends      896MB    22s  0.18s  rasterio GDAL 3.12.4, pyogrio GDAL 3.12.4
gdal base                       896MB    22s  0.18s  rasterio GDAL 3.12.4, pyogrio GDAL 3.12.4
gdal, multi-stage               836MB    22s  0.17s  rasterio GDAL 3.12.4, pyogrio GDAL 3.12.4

final image, largest layers:
    261MB  COPY /venv /venv # buildkit
   34.9MB  COPY /build_gdal_version_changing/usr/ /usr/…
   7.72MB  COPY /build_gdal_python/usr/ /usr/ # buildkit
   1.77MB  COPY /build/usr/include/ /usr/include/ # bui…
   2.92MB  COPY /build/usr/share/gdal/ /usr/share/gdal/…
   4.99MB  COPY /build/usr/local/gdal-internal/lib/ /us…
   10.3MB  COPY /build/usr/local/gdal-internal/share/pr…
   7.88MB  COPY /build_projgrids/usr/ /usr/ # buildkit
    235MB  RUN /bin/sh -c export DEBIAN_FRONTEND=nonint…
    100MB  umoci raw add-layer --image /home/buildd/roc…

Two separate wins, and they matter to different people. Choosing the base decides the size, for pushing, pulling and storage. Ordering the layers decides the rebuild, for whoever is developing, every day — and it comes entirely from where one COPY sits.

Example 3: verifying the image before shipping it

%%writefile test_image.py
"""Run against a built image: python -m pytest test_image.py"""
import os
import subprocess

import pytest

IMAGE = "sw-gis:final"


def run(code, *docker_args):
    result = subprocess.run(["docker", "run", "--rm", *docker_args, "--entrypoint", "python",
                             IMAGE, "-c", code], capture_output=True, text=True, timeout=120)
    if result.returncode != 0:
        pytest.fail(f"command failed:\n{result.stderr}")
    return result.stdout.strip()


def test_one_gdal():
    system = subprocess.run(["docker", "run", "--rm", "--entrypoint", "gdal-config", IMAGE, "--version"],
                            capture_output=True, text=True).stdout.strip()
    bound = run("import rasterio, pyogrio; print(rasterio.__gdal_version__, pyogrio.__gdal_version_string__)")
    assert bound == f"{system} {system}", f"bindings use {bound}, the image has {system}"


def test_proj_network_is_off_and_the_grid_is_present():
    out = run("import pyproj; from pyproj import Transformer;"
              "t = Transformer.from_crs(4326, 27700, always_xy=True);"
              "print(pyproj.network.is_network_enabled(), *t.transform(-2.2426, 53.4808))")
    enabled, x, y = out.split()
    assert enabled == "False", "PROJ_NETWORK must be off for reproducible transforms"
    assert abs(float(x) - 383_997.286) < 0.01 and abs(float(y) - 398_256.533) < 0.01, \
        "OSTN15 grid appears to be missing"


def test_runs_as_non_root():
    out = subprocess.run(["docker", "run", "--rm", "--entrypoint", "id", IMAGE, "-u"],
                         capture_output=True, text=True).stdout.strip()
    assert out == "10001"


def test_writes_to_a_mounted_volume_as_the_host_user(tmp_path):
    run("import geopandas as gpd; from shapely.geometry import box;"
        "gpd.GeoDataFrame({'id': [1]}, geometry=[box(0, 0, 1, 1)], crs=27700).to_file('/out/t.gpkg')",
        "--user", f"{os.getuid()}:{os.getgid()}", "-v", f"{tmp_path}:/out")
    assert (tmp_path / "t.gpkg").exists()


def test_image_is_not_absurdly_large():
    size = int(subprocess.run(["docker", "image", "inspect", IMAGE, "--format", "{{.Size}}"],
                              capture_output=True, text=True).stdout)
    assert size < 900 * 1024 ** 2, f"image is {size / 1024 ** 2:.0f} MB"

import sys

# --basetemp: temporary directories the Docker daemon can see and mount
!{sys.executable} -m pytest -q -p no:cacheprovider --basetemp=pytest-tmp test_image.py 2>&1 | tail -2

.....                                                                    [100%]
5 passed in 2.32s

The grid test is the valuable one. It asserts a transformed coordinate, so a missing grid — which produces no error, just a transformation 1.8 m out in this case — fails the build. Asserting library versions would not catch it, because the versions are fine; only the grid data is absent.

The volume test runs as the host user on purpose: as the image's own uid 10001, the same write failed (step 6). The size assertion is a ratchet. It does not make the image small, but it stops it quietly growing back.

And the claim this page used to make about memory, checked in the same image — GDAL's default block cache under no limit and under a 1 GB container limit:

def run(code, *docker_args):
    return sh("docker", "run", "--rm", *docker_args, "--entrypoint", "python", "sw-gis:final", "-c", code).strip()


probe = ("import ctypes, glob, os, rasterio;"
         "lib = ctypes.CDLL(glob.glob(os.path.join(os.path.dirname(rasterio.__file__), '..', 'rasterio.libs', 'libgdal*'))[0]"
         "      if glob.glob(os.path.join(os.path.dirname(rasterio.__file__), '..', 'rasterio.libs', 'libgdal*'))"
         "      else 'libgdal.so');"
         "lib.GDALGetCacheMax64.restype = lib.CPLGetUsablePhysicalRAM.restype = ctypes.c_int64;"
         "print(f'usable RAM {lib.CPLGetUsablePhysicalRAM() / 2**30:.1f} GB, default cache {lib.GDALGetCacheMax64() / 2**20:.0f} MB')")
for limit in ([], ["--memory=1g"]):
    print(" ".join(limit) or "no limit   ", run(probe, *limit))

sh("docker", "image", "rm", "sw-gis:final", "sw-gis:test")

no limit    usable RAM 60.9 GB, default cache 3119 MB
--memory=1g usable RAM 1.0 GB, default cache 51 MB

GDAL reads the container's memory limit: its default cache is 5% of the memory it can actually use. Set GDAL_CACHEMAX when a job needs more, not because the default ignores the container.

Why the image ends in C libraries

A Python GIS image is large for a specific reason: the dependency graph terminates in C libraries, not in Python. GEOS is a computational geometry engine, PROJ carries a database of coordinate systems, and GDAL supports a couple of hundred formats and links against libraries for each. There is no version of this stack that fits in 50 MB.

That fact explains the base-image decision. The naive image is large because python:3.12 ships a complete build toolchain — compilers, headers, development libraries — none of which is needed to run anything. Alpine fails because musl libc cannot use PyPI's glibc wheels, so anything without a musllinux wheel must compile from source, which needs the toolchain Alpine was chosen to avoid. The slim image works because the wheels carry their native libraries with them — so completely that a GDAL base image underneath them goes unused unless the bindings are compiled against it.

Multi-stage builds work because build-time and run-time dependencies barely overlap. Compiling needs gcc, python3-dev and header packages; executing needs the compiled artefacts and the shared libraries they link against. A stage boundary keeps only the second set, and a virtualenv makes the boundary crossable in one instruction. Without a compile step there is little to leave behind, which is why the wheels-only multi-stage image saved 60 MB and the compiled one saved more by dropping the wheels' second GDAL.

Layer caching is a separate concern from size, and often the more valuable one. Docker invalidates every layer after the first change, so an instruction's position determines how often it re-runs. Dependencies change monthly; source changes hourly. Putting the install above COPY src/ is the difference between 12 s and a fifth of a second here, and between two minutes and a fifth of a second for a stack that compiles.

Finally, an image is a reproducibility artefact or it is nothing. Its whole purpose is that the same bytes run the same way anywhere. That property is undone by a mutable base tag, an unpinned apt-get install, or a grid fetched at run time. Pinning the base by digest, hash-pinning the Python requirements, and building the grid into the image are what make it a fixed thing rather than a recipe that resolves differently each time — the argument developed in reproducible GIS environments explained.

Alpine, libexpat1 and digest pinning

• Alpine forces source builds of anything without a musllinux wheel — rasterio, here. Use a Debian or Ubuntu base.
• python:3.12-slim needs libexpat1 for rasterio's wheel; the build succeeds without it and the import fails.
• A GDAL base image is unused by wheels. Compile the bindings with --no-binary to share its GDAL, or use slim.
• rm -rf /var/lib/apt/lists/* must be in the same RUN as the apt-get install, or the files remain in the earlier layer.
• --mount=type=cache needs BuildKit, and does nothing if PIP_NO_CACHE_DIR is set in that stage.
• Pin the base image by digest for scheduled work; a tag can be re-pushed.
• pyproj's wheel reads its own data directory, so put grids there with pyproj sync; apt-get install proj-data does not reach it.
• A fixed non-root uid cannot write to a host directory it does not own — run with --user or set the directory's owner.
• GDAL honours the container's memory limit — its default cache was 51 MB under a 1 GB limit.
• Test stages should run python -m pytest, or the working directory is not importable.

Internal links

• How to containerise a Python GIS pipeline with Docker — the wider practice
• Reproducible GIS environments explained — why the native stack must be pinned
• GDAL and GeoPandas fail inside Docker — diagnosing a build that will not work
• How to set up a Python GIS environment that actually works — the same problem outside a container
• How to run a Python GIS pipeline in CI with GitHub Actions — building the image in CI
• GIS tests run out of memory or time out in CI — where GDAL_CACHEMAX matters
• What GDAL and OGR actually are — what all this weight is
• How to schedule a Python GIS script to run automatically — running the image

FAQ

Why is my Python GIS image so large?

Because GEOS, PROJ and GDAL are large C libraries, and a python:3.12 base adds a full build toolchain on top. Measured for the same two requirements, the naive image was 1.66 GB and python:3.12-slim with one extra library 538 MB.

Can I use Alpine to make it smaller?

Not for this stack. Alpine uses musl libc and PyPI's manylinux wheels target glibc; rasterio publishes no musllinux wheel, so pip tries to compile it and fails asking for gdal-config.

Is a GDAL base image better than python-slim?

Only if you compile the bindings against it. Installed from wheels, rasterio and pyogrio load their own GDAL 3.12.4 and the base's 3.13.3 goes unused — 896 MB with two GDALs. Compiled in a multi-stage build it is 668 MB with one; slim with wheels is 538 MB.

Why is my rebuild so slow after a one-line code change?

COPY . /app sits above the dependency install, so any file change invalidates it. Copy requirements.txt and install first, then copy the source — 12 s against 0.2 s here.

Does GDAL ignore the container's memory limit?

No. Measured, GDAL saw 1.0 GB of usable memory under a 1 GB limit and set its default block cache to 51 MB. Set GDAL_CACHEMAX to give a job more, not to protect the container.

How do I get PROJ's transformation grids into the image?

With pyproj sync --file <grid> --target-directory <pyproj's data dir> at build time. The pyproj wheel does not read the system's PROJ data, and with PROJ_NETWORK=OFF a missing grid silently falls back — 1.8 m out for Manchester in British National Grid.

Should I pin the base image by digest?

For anything scheduled or shared, yes. A tag such as 3.13.3 can be re-pushed with different content; a sha256: digest cannot.