How to Run a Python GIS Pipeline in CI with GitHub Actions

Problem statement

The pipeline is tested, containerised and scheduled — on one machine, by one person. Which means:

a change is only validated when someone remembers to run the tests locally
"works on my machine" is still the only evidence that it works
the nightly job runs from a laptop that is sometimes closed
nobody notices that geopandas 1.1 changed a default until the output is wrong
the deliverable is built by hand, so no two builds are quite the same

Continuous integration moves all of that onto a machine that starts clean every time. For GIS work it does something more specific too: it proves the pipeline runs in an environment that is not yours, which is the only real test of whether the dependency pinning is honest.

The friction is real, though. Geospatial dependencies are heavy, CI runners have no GDAL by default, spatial tests may need a database, and jobs that download data are slow and flaky.

Quick answer

Run the tests on every push, and the pipeline on a schedule, in the same container the server uses:

cache the Python environment so installs do not dominate the run
install GDAL via the container image or apt, not by compiling
run the fast test suite on pull requests, the full one on main
run the pipeline on a schedule: trigger and upload the output as an artifact
keep credentials in repository secrets, never in the workflow file

# .github/workflows/ci.yml
name: CI

on:
  push:
    branches: [main]
  pull_request:
  schedule:
    - cron: "30 2 * * *"        # 02:30 UTC — note: UTC, not your local time

jobs:
  test:
    runs-on: ubuntu-latest
    container: ghcr.io/osgeo/gdal:ubuntu-small-3.9.2
    steps:
      - uses: actions/checkout@v4

      - name: Install Python dependencies
        run: |
          apt-get update && apt-get install -y --no-install-recommends python3-pip
          pip3 install --break-system-packages -r requirements.txt -r requirements-dev.txt

      - name: Run tests
        run: pytest -q -m "not db"

      - name: Verify the geospatial stack
        run: |
          python3 -c "
          import geopandas as gpd
          from shapely.geometry import Point
          g = gpd.GeoDataFrame(geometry=[Point(-3.19, 55.95)], crs=4326).to_crs(27700)
          assert 300_000 < g.geometry.iloc[0].x < 400_000
          print('geopandas', gpd.__version__, 'ok')"

Running the job inside the GDAL image is the shortcut that avoids most geospatial CI pain: GDAL, GEOS and PROJ are already there, correctly matched.

What each trigger is for

Grid of CI triggers — pull request, push to main, schedule, manual — and what each should run. — Four triggers, four different jobs. Running everything on every push is why people turn CI off.

Step-by-step solution

Vertical steps: checkout, restore cache, install deps, lint, test, run pipeline, upload artifact. — Seven steps — caching and the smoke test are the two that make it usable day to day.

Get GDAL onto the runner

Three approaches, in order of preference:

# 1. run the job inside an image that already has it  ← best
jobs:
  test:
    runs-on: ubuntu-latest
    container: ghcr.io/osgeo/gdal:ubuntu-small-3.9.2

# 2. rely on wheels — geopandas, pyogrio, rasterio and shapely all ship binaries
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/setup-python@v5
        with: { python-version: "3.12", cache: pip }
      - run: pip install -r requirements.txt        # no system GDAL needed

# 3. install the distro packages, when you need the CLI tools too
      - run: |
          sudo apt-get update
          sudo apt-get install -y gdal-bin libgdal-dev
          echo "GDAL_VERSION=$(gdal-config --version)" >> "$GITHUB_ENV"

Option 2 is enough for a pure GeoPandas pipeline and is the fastest to set up; option 1 is right when the pipeline also shells out to ogr2ogr or gdalwarp, or when you want CI and production to share one image.

Cache the dependencies

Installing the geospatial stack takes a couple of minutes; caching makes it seconds.

      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
          cache: pip
          cache-dependency-path: requirements*.txt

      # cache test fixtures or reference data that rarely change
      - name: Cache reference data
        uses: actions/cache@v4
        with:
          path: data/ref
          key: refdata-${{ hashFiles('scripts/fetch_reference.py') }}-v1

      - name: Fetch reference data
        run: python scripts/fetch_reference.py        # a no-op on a cache hit

Include a version suffix (-v1) in cache keys you may need to bust manually — changing it is easier than deleting caches through the UI.

Split the fast job from the slow one

jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with: { python-version: "3.12", cache: pip }
      - run: pip install ruff
      - run: ruff check src tests
      - run: ruff format --check src tests

  test:
    needs: lint
    runs-on: ubuntu-latest
    container: ghcr.io/osgeo/gdal:ubuntu-small-3.9.2
    steps:
      - uses: actions/checkout@v4
      - run: |
          apt-get update && apt-get install -y --no-install-recommends python3-pip
          pip3 install --break-system-packages -r requirements.txt -r requirements-dev.txt
      - run: pytest -q -m "not db" --junitxml=report.xml
      - uses: actions/upload-artifact@v4
        if: always()
        with: { name: test-report, path: report.xml }

Linting first fails a badly formatted pull request in fifteen seconds instead of after a four-minute test run.

Add PostGIS as a service when you need it

  test-db:
    runs-on: ubuntu-latest
    services:
      postgis:
        image: postgis/postgis:16-3.4
        env:
          POSTGRES_DB: gis
          POSTGRES_USER: gis
          POSTGRES_PASSWORD: gis
        ports: ["5432:5432"]
        options: >-
          --health-cmd "pg_isready -U gis"
          --health-interval 5s --health-timeout 5s --health-retries 10
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with: { python-version: "3.12", cache: pip }
      - run: pip install -r requirements.txt -r requirements-dev.txt
      - name: Run database tests
        env:
          PGHOST: localhost
          PGUSER: gis
          PGPASSWORD: gis
          PGDATABASE: gis
        run: pytest -q -m db

The health check is what removes the "connection refused on the first run" flake — the steps do not start until Postgres is actually accepting connections.

Test across versions where it matters

  matrix:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        python: ["3.11", "3.12", "3.13"]
        geopandas: ["1.0.1", "1.1.0"]
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with: { python-version: "${{ matrix.python }}", cache: pip }
      - run: |
          pip install -r requirements-dev.txt
          pip install "geopandas==${{ matrix.geopandas }}"
      - run: pytest -q -m "not db"

fail-fast: false matters here: you want to see which combinations broke, not just that one did.

Run the pipeline on a schedule and keep the output

  nightly:
    if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'
    runs-on: ubuntu-latest
    container: ghcr.io/osgeo/gdal:ubuntu-small-3.9.2
    timeout-minutes: 60
    steps:
      - uses: actions/checkout@v4

      - run: |
          apt-get update && apt-get install -y --no-install-recommends python3-pip
          pip3 install --break-system-packages -r requirements.txt

      - name: Run the pipeline
        env:
          PGHOST: ${{ secrets.PGHOST }}
          PGUSER: ${{ secrets.PGUSER }}
          PGPASSWORD: ${{ secrets.PGPASSWORD }}
          PGDATABASE: ${{ secrets.PGDATABASE }}
          GIS_DATA_ROOT: ${{ github.workspace }}/data
        run: python3 -m src.pipeline --config configs/daily.yml

      - name: Upload outputs
        uses: actions/upload-artifact@v4
        with:
          name: pipeline-output-${{ github.run_number }}
          path: |
            data/out/**
            logs/runs/*.json
          retention-days: 30

      - name: Summarise the run
        if: always()
        run: |
          python3 - <<'PY' >> "$GITHUB_STEP_SUMMARY"
          import json, glob
          latest = sorted(glob.glob("logs/runs/*.json"))[-1]
          rec = json.load(open(latest))
          print(f"### Run `{rec['run_id']}` — {rec['status']}\n")
          print(f"- duration: {rec['duration_s']}s")
          for o in rec["outputs"]:
              print(f"- output: `{o['path']}` — {o.get('features', '?')} features")
          PY

$GITHUB_STEP_SUMMARY renders Markdown on the run's page, so the feature counts are visible without downloading anything.

Add a scheduled-workflow keepalive and alerting

      - name: Notify on failure
        if: failure()
        uses: actions/github-script@v7
        with:
          script: |
            github.rest.issues.create({
              owner: context.repo.owner,
              repo: context.repo.repo,
              title: `Nightly pipeline failed (run ${context.runNumber})`,
              body: `The scheduled pipeline failed.\n\n${context.serverUrl}/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}`,
              labels: ["pipeline", "failure"],
            })

Two scheduling facts worth knowing: cron in Actions is UTC, and scheduled workflows are disabled automatically after 60 days without repository activity — so a quiet repo silently stops running its nightly job.

Code examples

Example 1: a complete workflow file

name: GIS pipeline

on:
  push: { branches: [main] }
  pull_request:
  schedule: [{ cron: "30 2 * * *" }]
  workflow_dispatch:
    inputs:
      config:
        description: Config file to run
        default: configs/daily.yml

concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

env:
  PYTHON_VERSION: "3.12"

jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with: { python-version: "3.12", cache: pip }
      - run: pip install ruff
      - run: ruff check src tests && ruff format --check src tests

  test:
    needs: lint
    runs-on: ubuntu-latest
    services:
      postgis:
        image: postgis/postgis:16-3.4
        env: { POSTGRES_DB: gis, POSTGRES_USER: gis, POSTGRES_PASSWORD: gis }
        ports: ["5432:5432"]
        options: >-
          --health-cmd "pg_isready -U gis" --health-interval 5s --health-retries 10
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with: { python-version: "3.12", cache: pip }
      - run: pip install -r requirements.txt -r requirements-dev.txt
      - name: Smoke-test the geospatial stack
        run: |
          python -c "
          import geopandas as gpd, pyproj, shapely
          from shapely.geometry import Point
          g = gpd.GeoDataFrame(geometry=[Point(-3.19, 55.95)], crs=4326).to_crs(27700)
          assert 300_000 < g.geometry.iloc[0].x < 400_000
          print('geopandas', gpd.__version__, '| proj', pyproj.proj_version_str)"
      - name: Tests
        env: { PGHOST: localhost, PGUSER: gis, PGPASSWORD: gis, PGDATABASE: gis }
        run: pytest -q --junitxml=report.xml --cov=src --cov-report=term-missing
      - uses: actions/upload-artifact@v4
        if: always()
        with: { name: test-report, path: report.xml }

  pipeline:
    needs: test
    if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'
    runs-on: ubuntu-latest
    container: ghcr.io/osgeo/gdal:ubuntu-small-3.9.2
    timeout-minutes: 90
    steps:
      - uses: actions/checkout@v4
      - run: |
          apt-get update && apt-get install -y --no-install-recommends python3-pip
          pip3 install --break-system-packages -r requirements.txt
      - name: Run
        env:
          PGHOST: ${{ secrets.PGHOST }}
          PGUSER: ${{ secrets.PGUSER }}
          PGPASSWORD: ${{ secrets.PGPASSWORD }}
          PGDATABASE: ${{ secrets.PGDATABASE }}
        run: python3 -m src.pipeline --config "${{ inputs.config || 'configs/daily.yml' }}"
      - uses: actions/upload-artifact@v4
        with:
          name: output-${{ github.run_number }}
          path: |
            data/out/**
            logs/runs/*.json
          retention-days: 30

concurrency with cancel-in-progress stops three pushes in five minutes from queueing three full runs.

Example 2: build and publish the container image

  image:
    needs: test
    if: github.ref == 'refs/heads/main'
    runs-on: ubuntu-latest
    permissions: { contents: read, packages: write }
    steps:
      - uses: actions/checkout@v4
      - uses: docker/setup-buildx-action@v3
      - uses: docker/login-action@v3
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}
      - uses: docker/metadata-action@v5
        id: meta
        with:
          images: ghcr.io/${{ github.repository }}
          tags: |
            type=raw,value=latest
            type=sha,format=long
            type=raw,value={{date 'YYYYMMDD'}}
      - uses: docker/build-push-action@v6
        with:
          context: .
          push: true
          tags: ${{ steps.meta.outputs.tags }}
          cache-from: type=gha
          cache-to: type=gha,mode=max

A date-tagged, immutable image per merge is what the scheduled server job should pin to.

Example 3: fail the build when the output changes unexpectedly

      - name: Check output against the golden summary
        run: |
          python3 - <<'PY'
          import json, sys
          import geopandas as gpd

          gdf = gpd.read_file("data/out/parcels.gpkg")
          actual = {
              "features": len(gdf),
              "crs": gdf.crs.to_string(),
              "bounds": [round(float(v), 1) for v in gdf.total_bounds],
          }
          expected = json.load(open("tests/golden/parcels_summary.json"))
          if actual != expected:
              print("::error::pipeline output changed")
              print(f"expected: {expected}")
              print(f"actual  : {actual}")
              sys.exit(1)
          print("output matches golden summary")
          PY

::error:: renders as a proper annotation on the run, so the reason is visible without opening the log.

Example 4: a manual re-run with parameters

on:
  workflow_dispatch:
    inputs:
      region:
        description: Region to process
        type: choice
        options: [all, north, south, east, west]
        default: all
      dry_run:
        description: Plan only, write nothing
        type: boolean
        default: true

jobs:
  run:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: |
          python -m src.pipeline \
            --region "${{ inputs.region }}" \
            ${{ inputs.dry_run && '--dry-run' || '' }}

Combining workflow_dispatch with a dry-run default gives non-developers a safe button to press.

Explanation

CI is a second machine that has never seen your laptop. That is the whole value: it starts from a clean checkout and a clean environment, so anything that only worked because of something installed years ago fails immediately, in public, with a log.

Flow from commit through test, build image, run pipeline, to artifacts and alerts. — One commit, four outcomes — the artifacts are what turn CI into a delivery mechanism.

For geospatial work the environment question is sharper than for ordinary Python, because the dependencies extend below the language. The two robust answers are the same as everywhere else in this stack: rely on wheels, which bundle their own GDAL and are enough for a GeoPandas pipeline, or run the job inside the same GDAL image the pipeline uses in production. Compiling GDAL in CI is possible and almost never worth the ten minutes per run.

Structuring the workflow around triggers keeps it fast enough to stay enabled. A pull request wants lint plus the fast tests — under two minutes, or people start merging without waiting. A push to main can afford the database tests and an image build. A schedule runs the actual pipeline. A workflow_dispatch gives you a button for re-runs and one-off parameters. Bundling all of that into one job that runs on every event is the most common reason teams end up ignoring a permanently red CI.

Artifacts are what make CI more than a test runner for a data pipeline. Uploading the output and the run record turns each scheduled execution into something reviewable: the feature counts are in the step summary, the GeoPackage is downloadable for thirty days, and a golden-summary check fails the run when the numbers move unexpectedly. That is a data-quality gate, and it costs a few lines of YAML.

Two operational details bite people specifically with scheduled workflows. Cron in Actions is UTC, so a "02:30" job runs an hour off your local time for half the year. And GitHub disables scheduled workflows in repositories with no activity for 60 days — a quiet pipeline repo stops running without any notification at all, which is worth a calendar reminder or a periodic commit.

Edge cases or notes

Schedules are UTC and imprecise: Jobs queue at busy times and may start several minutes late. Do not depend on the exact minute.
Scheduled workflows are auto-disabled after 60 days of inactivity: Push something occasionally, or re-enable it manually.
GITHUB_TOKEN permissions default to read-only: Grant packages: write or contents: write explicitly per job when you need them.
Artifacts are not free: They count against storage. Set retention-days, and do not upload multi-gigabyte rasters routinely.
Runners are ephemeral and modest: Two cores and 7 GB of RAM on the standard Linux runner. Large geospatial jobs need chunking or a self-hosted runner.
Container jobs run as root: Files created in the workspace are root-owned, which matters if a later step runs as another user.
pull_request from a fork has no secrets: By design. Split jobs so fork PRs run the tests that do not need credentials.

Internal links

FAQ

How do I get GDAL onto a GitHub Actions runner?

Easiest: run the job inside ghcr.io/osgeo/gdal:ubuntu-small-<version>. Alternatively rely on the wheels — geopandas, pyogrio, rasterio and shapely all ship binaries — or apt-get install gdal-bin libgdal-dev when you need the CLI tools.

Should the pipeline itself run in CI?

For small to medium jobs, yes: a scheduled workflow plus uploaded artifacts is a complete, free scheduler with logs and alerting. For long or memory-hungry jobs, use CI for tests and images, and run the pipeline on your own infrastructure from a pinned image.

How do I test against PostGIS?

Add postgis/postgis as a service container with a pg_isready health check, and pass the connection settings as environment variables. The health check is what stops the first-run connection flake.

Why did my scheduled workflow stop running?

Either the repository has had no activity for 60 days, in which case GitHub disabled it, or the cron is in UTC and running at a different local time than you expected.

How do I keep CI fast?

Cache pip and any downloaded reference data, split lint from tests, run only the fast tests on pull requests, and use concurrency with cancel-in-progress so superseded runs stop immediately.

Where do credentials go?

Repository or environment secrets, referenced as $ and passed via env: on the step that needs them. Never in the workflow file, and remember that pull requests from forks do not receive secrets.

How do I know what the nightly run produced?

Write a run record, upload it and the outputs as an artifact, and print a summary into $GITHUB_STEP_SUMMARY so the counts appear on the run page without downloading anything.

How to Run a Python GIS Pipeline in CI with GitHub Actions #

Problem statement #

Quick answer #

What each trigger is for #

Step-by-step solution #

Get GDAL onto the runner #

Cache the dependencies #

Split the fast job from the slow one #

Add PostGIS as a service when you need it #

Test across versions where it matters #

Run the pipeline on a schedule and keep the output #

Add a scheduled-workflow keepalive and alerting #

Code examples #

Example 1: a complete workflow file #

Example 2: build and publish the container image #

Example 3: fail the build when the output changes unexpectedly #

Example 4: a manual re-run with parameters #

Explanation #

Edge cases or notes #

Internal links #

FAQ #

How do I get GDAL onto a GitHub Actions runner? #

Should the pipeline itself run in CI? #

How do I test against PostGIS? #

Why did my scheduled workflow stop running? #

How do I keep CI fast? #

Where do credentials go? #

How do I know what the nightly run produced? #