⚙️ How-To

How to Schedule a Python GIS Script to Run Automatically

The script works. You run it every Monday morning, it takes eleven minutes, and it produces the layer everyone downstream depends on. The obvious next step is to stop being the thing that triggers it — and that is where most people meet the oldest trap in automation: a script that runs perfectly in a terminal and fails silently under a scheduler, because the scheduler gives it a different working directory, a different PATH, no virtual environment, and nowhere to send its output. This article covers scheduling properly on Linux, macOS, and Windows, and the handful of details that make the difference between a job that runs for a year and one that quietly stopped in March.

Problem statement

You want a GIS script to run on a schedule without you. The obstacles:

  • It works interactively, not from cron — ModuleNotFoundError: No module named 'geopandas', even though it imports fine in your shell.
  • Relative paths resolve somewhere else — the job runs with a working directory you did not expect and writes output into a surprising folder.
  • Output vanishes — print() goes nowhere, so a failure leaves no trace.
  • Overlapping runs — a slow run is still going when the next one starts, and two processes write the same file.
  • Nobody notices failures — the job stopped six weeks ago and the stale layer looked plausible enough that nobody checked.

The goal: a scheduled job with an absolute interpreter path, an explicit working directory, a log file per run, a lock so runs cannot overlap, and a signal when something breaks.

Quick answer

Point the scheduler at the interpreter inside your virtual environment using absolute paths, set the working directory explicitly, and redirect both stdout and stderr to a log file. On Linux and macOS that is one crontab line; on Windows it is one schtasks command.

The five cron time fields — minute, hour, day of month, month, day of week — with an example schedule.
Five fields, left to right, smallest unit first; an asterisk means "every".
# crontab -e   →   every day at 02:15
15 2 * * * cd /srv/gis/parcels && /srv/gis/parcels/.venv/bin/python run_pipeline.py \
  --config config/prod.yml >> /var/log/gis/parcels.log 2>&1
# Windows Task Scheduler, every day at 02:15
schtasks /Create /TN "GIS parcels pipeline" /SC DAILY /ST 02:15 ^
  /TR "C:\gis\parcels\.venv\Scripts\python.exe C:\gis\parcels\run_pipeline.py --config C:\gis\parcels\config\prod.yml" ^
  /RL HIGHEST

Three details do most of the work: the absolute path to the venv's Python, the explicit cd into the project, and >> log 2>&1 so both normal output and errors are captured.

Step-by-step solution

Make the script runnable without you first

Before scheduling anything, the script must take everything it needs from arguments or a config file, write its own log, and exit with a meaningful status code. A script that requires an interactive decision cannot be scheduled at all. If yours still has constants at the top, start with turning it into a command-line tool.

# it must work from anywhere, not just from the project folder
cd /tmp && /srv/gis/parcels/.venv/bin/python /srv/gis/parcels/run_pipeline.py --config /srv/gis/parcels/config/prod.yml

If that command fails, scheduling it will fail too — in a harder-to-debug way.

Use the virtual environment's interpreter directly

You do not need to "activate" a virtualenv in a scheduled job. Activation only puts a directory on PATH; calling the interpreter inside the venv by absolute path achieves the same thing with nothing to go wrong.

/srv/gis/parcels/.venv/bin/python    # Linux, macOS
C:\gis\parcels\.venv\Scripts\python.exe   # Windows

This single change fixes the great majority of "works in my terminal, not in cron" failures. Cron runs with a minimal environment — often just /usr/bin:/bin — so a bare python resolves to the system interpreter, which has never heard of GeoPandas.

Set the working directory explicitly

Cron starts jobs in the user's home directory; Task Scheduler uses C:\Windows\System32 unless told otherwise. Any relative path in your script or config resolves against that. Either cd into the project first, or make every path in the job absolute.

15 2 * * * cd /srv/gis/parcels && .venv/bin/python run_pipeline.py --config config/prod.yml >> logs/parcels.log 2>&1

Better still, remove the dependency entirely by resolving config paths against the config file's own location, as shown in driving a pipeline from YAML.

Capture output — all of it

>> file redirects stdout; 2>&1 sends stderr to the same place. Without the second part, tracebacks disappear. Use >> (append) rather than > (truncate) so one run does not erase the evidence of the last.

>> /var/log/gis/parcels.log 2>&1

Better than shell redirection is having the script log to a file itself, with timestamps and levels, and keeping the redirect only as a safety net for anything that escapes the logger — see log and summarise errors in a batch GIS job.

import logging
from logging.handlers import RotatingFileHandler

handler = RotatingFileHandler("logs/parcels.log", maxBytes=5_000_000, backupCount=5)
handler.setFormatter(logging.Formatter("%(asctime)s %(levelname)-7s %(message)s"))
logging.basicConfig(level=logging.INFO, handlers=[handler])

Rotation matters: an unrotated log on a daily job will eventually fill the disk, and a full disk fails every job on the machine, not just yours.

Stop runs from overlapping

If a run occasionally takes longer than the interval, two copies will eventually run at once — both writing the same output. A lock file prevents it.

import fcntl, sys
from pathlib import Path

LOCK = Path("/tmp/parcels-pipeline.lock")

def acquire_lock():
    fh = LOCK.open("w")
    try:
        fcntl.flock(fh, fcntl.LOCK_EX | fcntl.LOCK_NB)
    except BlockingIOError:
        sys.exit("another run is still in progress; exiting")
    return fh          # keep the handle alive for the process lifetime

On Linux, flock(1) does the same thing from the crontab line without touching the script:

15 2 * * * /usr/bin/flock -n /tmp/parcels.lock -c 'cd /srv/gis/parcels && .venv/bin/python run_pipeline.py' >> logs/parcels.log 2>&1

Windows Task Scheduler has this built in: set the task to not start a new instance if one is already running.

Choose the right scheduler for the machine

Comparison of cron, systemd timers, Windows Task Scheduler, and a cloud scheduler across setup, logging, and reliability.
All four run a command on a clock; they differ in what they give you when it fails.

cron (Linux, macOS) — simplest, everywhere, no logging or failure handling of its own. systemd timers (modern Linux) — more setup, but real logs via journalctl, dependency handling, and Persistent=true to catch up on a missed run after a reboot. Task Scheduler (Windows) — a GUI or schtasks, run-as-user configuration, and built-in "do not start if already running". Cloud/CI schedulers (GitHub Actions, Cloud Scheduler, Airflow) — right when the machine itself should not be a dependency.

A systemd unit and timer, for a job that deserves proper logging:

# /etc/systemd/system/gis-parcels.service
[Unit]
Description=Parcels GIS pipeline

[Service]
Type=oneshot
User=gis
WorkingDirectory=/srv/gis/parcels
ExecStart=/srv/gis/parcels/.venv/bin/python run_pipeline.py --config config/prod.yml
# /etc/systemd/system/gis-parcels.timer
[Unit]
Description=Run the parcels pipeline daily

[Timer]
OnCalendar=*-*-* 02:15:00
Persistent=true

[Install]
WantedBy=timers.target
sudo systemctl daemon-reload
sudo systemctl enable --now gis-parcels.timer
systemctl list-timers gis-parcels.timer     # confirm the next run time
journalctl -u gis-parcels.service -n 50     # read the last run's output

Know that it ran

A scheduled job with no monitoring is a job you will one day discover has been dead for months. The cheapest useful check is a "heartbeat": write a timestamp file on success and have something look at its age. Proper alerting is covered in get alerted when an automated GIS job fails.

import json
from datetime import datetime, timezone
from pathlib import Path

def heartbeat(ok=True):
    state = {"at": datetime.now(timezone.utc).isoformat(timespec="seconds"), "ok": ok}
    Path("logs/last_run.json").write_text(json.dumps(state))

Code examples

Example 1: Common cron schedules 

# every day at 02:15
15 2 * * * …
# every Monday at 06:00
0 6 * * 1 …
# every 15 minutes
*/15 * * * * …
# first day of every month at 03:30
30 3 1 * * …
# weekdays only, every two hours during office hours
0 9-17/2 * * 1-5 …

Test an expression before trusting it — an off-by-one in the day-of-week field is easy and invisible for a week.

Example 2: A wrapper script instead of a long crontab line

Keeping the logic in a shell script makes the crontab readable and lets you run exactly what cron runs, by hand.

#!/usr/bin/env bash
# /srv/gis/parcels/run.sh
set -euo pipefail            # fail on error, unset variable, or failed pipe

PROJECT=/srv/gis/parcels
LOG="$PROJECT/logs/parcels-$(date +%F).log"
mkdir -p "$PROJECT/logs"

cd "$PROJECT"
echo "=== start $(date -Is) ===" >> "$LOG"
"$PROJECT/.venv/bin/python" run_pipeline.py --config config/prod.yml >> "$LOG" 2>&1
status=$?
echo "=== end $(date -Is) status=$status ===" >> "$LOG"
exit $status
15 2 * * * /srv/gis/parcels/run.sh

Example 3: Making the script defend itself

Anything a scheduler can get wrong, the script can check at startup.

import os, sys
from pathlib import Path

def preflight(cfg):
    project = Path(__file__).resolve().parent
    os.chdir(project)                      # never trust the inherited cwd
    if sys.prefix == sys.base_prefix:
        sys.exit("error: not running inside the virtual environment")
    for name, path in cfg["input"].items():
        if not Path(path).exists():
            sys.exit(f"error: input '{name}' missing: {path}")
    free_gb = os.statvfs(cfg["output"]["dir"]).f_bavail * 4096 / 1e9
    if free_gb < 2:
        sys.exit(f"error: only {free_gb:.1f} GB free on the output volume")

Example 4: Windows Task Scheduler with XML

The GUI is fine for one task; XML is what you commit to a repository.

schtasks /Create /TN "GIS\parcels-daily" /XML C:\gis\parcels\task.xml
schtasks /Run /TN "GIS\parcels-daily"        # test it immediately
schtasks /Query /TN "GIS\parcels-daily" /V /FO LIST   # last result & run time

The value to check after a test run is Last Result: 0 means success, and anything else maps to the exit code your script returned.

Example 5: A GitHub Actions schedule, when the data is remote

If the inputs and outputs live in cloud storage, the job does not need your server at all.

name: parcels-pipeline
on:
  schedule:
    - cron: "15 2 * * *"     # UTC — always UTC on GitHub Actions
  workflow_dispatch:          # lets you run it by hand too

jobs:
  run:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - run: pip install -r requirements.txt
      - run: python run_pipeline.py --config config/ci.yml
        env:
          PARCELS_DB_PASSWORD: ${{ secrets.PARCELS_DB_PASSWORD }}

A failed run emails you by default, which is more monitoring than most cron jobs ever get.

Explanation

The reason scheduled jobs fail in ways interactive runs never do is that a scheduler deliberately gives your process an empty context. Your shell has spent years accumulating a PATH, an activated environment, a working directory, and a terminal to print to. Cron gives you almost none of that, on purpose, so that jobs are reproducible rather than dependent on whoever last logged in. Every fix in this article is really the same fix: state explicitly what your shell was providing implicitly.

Four scheduling failures and their fixes: wrong interpreter, wrong working directory, lost output, and overlapping runs.
Four failures account for most broken cron jobs — and all four have one-line fixes.

Time zones deserve real care in GIS work, because the data usually has a date in it. Cron uses the machine's local time zone, so a daily 02:15 job on a server that observes daylight saving will run twice on one night in autumn and not at all on one night in spring. If your pipeline stamps outputs with a date, that is a duplicate or a gap in your data. Run scheduled jobs on machines set to UTC, and if a run must happen at a local-time boundary, compute that boundary inside the script where you can see the logic.

Idempotence is what turns scheduling from fragile to boring. A job is idempotent if running it twice with the same inputs leaves the same result as running it once. Writing into a dated run folder, refusing to overwrite without --force, and deriving output names from inputs all push in that direction. Once a job is idempotent, "did it run twice?" stops being a question that matters, and rerunning after a failure becomes safe rather than nerve-racking.

The last piece is knowing the job is alive. Silence from a scheduler means nothing at all — it is the same signal for "ran perfectly" and "the machine rebooted in March". A heartbeat file, a summary line in a log you actually read, or a proper failure alert turns silence into information. Pick one before you schedule anything that matters.

Edge cases or notes

macOS: cron works, but launchd is native

Cron still runs on macOS, though it needs Full Disk Access granted to /usr/sbin/cron for jobs touching protected folders. launchd with a .plist in ~/Library/LaunchAgents is the supported mechanism and handles laptop sleep better — a laptop that was closed at 02:15 simply never runs a cron job, while launchd can run it on wake.

The job runs as a different user

Cron entries belong to a user, and the system crontab has an extra user field. A job that works under your account may fail under a service account because of file permissions, a different home directory, or no access to a network share. Test with sudo -u gis /srv/gis/parcels/run.sh rather than assuming.

Percent signs in crontab

In a crontab line, an unescaped % is turned into a newline. date +%F inside a crontab command therefore breaks in a mystifying way. Escape it as \%, or — better — put the command in a wrapper script where normal shell rules apply.

The machine reboots or sleeps

Plain cron has no memory: a job scheduled while the machine was off simply does not run. systemd timers with Persistent=true run the missed job on the next boot, and anacron covers laptops. If a missed run matters, choose a mechanism that catches up.

Long-running jobs and the next tick

A job that sometimes takes three hours on a two-hourly schedule will eventually overlap itself. Use a lock, and log the duration of every run so you can see the trend before it becomes a collision. If runtime is creeping up, parallel processing or caching unchanged work usually buys more headroom than a longer interval.

Credentials in the scheduled environment

Interactive shells load ~/.bashrc; cron does not. Environment variables holding database passwords will simply be absent. Set them in the crontab itself (readable only by that user), in a systemd unit's EnvironmentFile=, or in a .env file the script loads deliberately — and never in a file that is committed.

FAQ

Why does my script work in the terminal but fail under cron?

Almost always because cron runs with a minimal environment. Your terminal has an activated virtualenv and a rich PATH; cron has neither, so a bare python resolves to the system interpreter without GeoPandas installed. Call the venv's interpreter by absolute path — /srv/gis/parcels/.venv/bin/python — and the problem disappears.

Do I need to activate the virtual environment in a cron job?

No. Activation is only a convenience that adjusts PATH for an interactive shell. Invoking .venv/bin/python directly gives you exactly the same interpreter and packages, with nothing to forget. Use sys.prefix != sys.base_prefix at startup if you want the script to assert it is running in the right environment.

Where does the output of a scheduled script go?

Nowhere, unless you redirect it. Append both streams to a file with >> /path/to.log 2>&1, and preferably have the script write its own timestamped, rotating log as well. Without the 2>&1, tracebacks go to stderr and are lost, which is how failures become invisible.

How do I stop two runs from overlapping?

Take an exclusive lock. On Linux, wrap the command in flock -n /tmp/job.lock -c '…' so a second run exits immediately; in Python, use fcntl.flock on a lock file kept open for the process's lifetime. Windows Task Scheduler has a setting for it. Without a lock, two processes will eventually write the same output file simultaneously.

Should I use cron or a systemd timer?

Cron is fine for a simple job on a machine that is always on. Prefer a systemd timer when you want logs in journalctl, a catch-up run after downtime (Persistent=true), dependencies on other units, or a clean way to check when the job last ran and what it returned. The extra setup is two short files.

How do I handle time zones and daylight saving?

Run scheduled jobs on a machine set to UTC. On a machine observing daylight saving, a job scheduled in the shifted hour runs twice in autumn and is skipped in spring — which corrupts any pipeline that stamps outputs by date. If a run must align with local time, compute the local boundary inside the script.

How will I know if the scheduled job stops running?

You will not, unless you arrange it. Silence from a scheduler is indistinguishable from success. Write a heartbeat file on every successful run and check its age, or send an explicit notification on failure — see get alerted when an automated GIS job fails. Do this before scheduling anything anyone depends on.