Python GIS Script Works Manually but Not from Cron: How to Fix It

Problem statement

The script runs perfectly from your terminal. Scheduled, it produces nothing — no output file, no log line, and often no error anywhere you thought to look.

$ python update_parcels.py
wrote data/out/parcels_2026-08-11.gpkg in 42s

# crontab entry
30 2 * * * python update_parcels.py
# … next morning: no new file, no message

Cron did run it. What it ran was a different environment: a different PATH, a different working directory, no virtualenv, no shell profile, and no terminal to print to. Anything your script inherited from your interactive session is simply absent.

Common causes:

python on cron's PATH is the system interpreter, not your virtualenv
the working directory is the user's home, so every relative path misses
environment variables set in .bashrc or .profile are never loaded
output goes to the void because cron mails stdout and no mailer is configured
the crontab line has an unescaped %, which cron treats as a newline
the file is not executable, or the shebang points at the wrong interpreter
GDAL/PROJ environment variables (PROJ_LIB, GDAL_DATA) are unset

The same list applies almost unchanged to Windows Task Scheduler, systemd timers, and Airflow workers. The difference is never "cron is broken" — it is that an interactive shell does a lot of setup you never see.

Quick answer

To make a scheduled GIS script behave like the manual run:

use absolute paths for the interpreter, the script, and every data path
redirect stdout and stderr to a log file so failures leave a trace
set the working directory explicitly in the script, not in the crontab
escape % as \% in crontab lines
run the exact cron command by hand with env -i to reproduce the failure

# crontab -e
30 2 * * * cd /srv/gis && /srv/gis/.venv/bin/python /srv/gis/update_parcels.py >> /srv/gis/logs/parcels.log 2>&1

Three absolute paths and one redirect fix the large majority of cases. .venv/bin/python runs the script with the virtualenv's packages without any activation step — that is what a venv's interpreter does.

What cron does not give you

Comparison grid of interactive shell versus cron environment across PATH, cwd, env vars and output. — Four differences account for nearly every "works manually, fails scheduled" report.

Step-by-step solution

Vertical steps for debugging a scheduled script: capture output, print environment, reproduce with env -i, fix. — Debug by making the failure visible first — guessing at causes wastes days of overnight runs.

Capture the output before anything else

A scheduled job that fails invisibly cannot be debugged. Redirect both streams to a file, appending so you keep the history.

30 2 * * * /srv/gis/.venv/bin/python /srv/gis/update_parcels.py >> /srv/gis/logs/parcels.log 2>&1

2>&1 must come after the >>, or stderr still goes to cron's mail. Add a timestamp per run so a log covering weeks stays readable:

30 2 * * * { date -Is; /srv/gis/.venv/bin/python /srv/gis/update_parcels.py; echo "exit=$?"; } >> /srv/gis/logs/parcels.log 2>&1

On systemd, output goes to the journal automatically — journalctl -u parcels.service. On Windows Task Scheduler, the task history pane records the exit code, and you still want a redirect for the text.

Find out which interpreter cron is using

The single most common failure is the wrong Python. Have the script report its own environment on start-up.

import sys, os
from pathlib import Path

print("python  :", sys.executable)
print("version :", sys.version.split()[0])
print("cwd     :", Path.cwd())
print("user    :", os.environ.get("USER") or os.environ.get("USERNAME"))
print("PATH    :", os.environ.get("PATH"))

Run it manually, then run it scheduled, and compare the two logs. If sys.executable is /usr/bin/python3 under cron and /srv/gis/.venv/bin/python3 in your terminal, you have found the problem — and the fix is to call the venv interpreter by absolute path.

Never rely on `activate`

Activating a virtualenv is a shell convenience; it only edits PATH. Cron does not run your shell profile, so there is nothing to activate into. Two reliable options:

# option 1 — call the venv interpreter directly (preferred)
/srv/gis/.venv/bin/python /srv/gis/update_parcels.py

# option 2 — run a login shell that sources the profile, then activate
0 2 * * * /bin/bash -lc 'source /srv/gis/.venv/bin/activate && python /srv/gis/update_parcels.py'

Option 1 has fewer moving parts, and it keeps working when the profile changes. Use option 2 only when the script genuinely needs variables that live in the profile.

Set the working directory in the script

Relative paths resolve against the process working directory, which cron sets to the user's home. Anchoring inside the script means the job behaves identically no matter who starts it or from where.

from pathlib import Path
import os

BASE = Path(__file__).resolve().parent
os.chdir(BASE)                    # optional, but makes relative paths predictable

SRC = BASE / "data" / "raw"
OUT = BASE / "data" / "out"
OUT.mkdir(parents=True, exist_ok=True)

Alternatively cd /srv/gis && … in the crontab line. Doing it in the script is more robust, because it also covers manual runs from an unexpected directory.

Provide the environment variables the libraries need

GDAL, PROJ and some cloud drivers read configuration from the environment. Conda installations in particular set PROJ_LIB and GDAL_DATA during activation.

import os
os.environ.setdefault("PROJ_LIB", "/srv/gis/.venv/share/proj")
os.environ.setdefault("GDAL_DATA", "/srv/gis/.venv/share/gdal")

Better: keep the job's variables in one file and load it, so the same values are used by every entry point.

# /srv/gis/job.env
PROJ_LIB=/srv/gis/.venv/share/proj
GDAL_DATA=/srv/gis/.venv/share/gdal
DB_URL=postgresql://gis:[email protected]/parcels

30 2 * * * set -a && . /srv/gis/job.env && set +a && /srv/gis/.venv/bin/python /srv/gis/update_parcels.py >> /srv/gis/logs/parcels.log 2>&1

Cron also supports plain KEY=value lines at the top of a crontab, but those are not expanded like shell variables — PATH=$PATH:/usr/local/bin does not work there.

Escape percent signs

In a crontab, an unescaped % becomes a newline, and everything after it is fed to the command as standard input. Any date format string will hit this.

# broken: cron truncates at the first %
0 2 * * * /srv/gis/run.sh --date $(date +%Y-%m-%d)

# correct
0 2 * * * /srv/gis/run.sh --date $(date +\%Y-\%m-\%d)

This is a good reason to keep crontab lines trivial and put the logic in a wrapper script, where no escaping is needed.

Reproduce the failure on demand

You do not have to wait until 02:30 to test. Strip the environment and run the exact command:

env -i /bin/sh -c 'cd $HOME && /srv/gis/.venv/bin/python /srv/gis/update_parcels.py'

env -i starts with an empty environment, which is close to what cron provides. If it fails here, you can iterate in seconds instead of nights.

Code examples

Example 1: a wrapper script that makes the job self-contained

#!/usr/bin/env bash
# /srv/gis/run_parcels.sh
set -euo pipefail

BASE="/srv/gis"
LOG="$BASE/logs/parcels-$(date +%Y%m%d).log"

cd "$BASE"
set -a; . "$BASE/job.env"; set +a

{
  echo "=== start $(date -Is) ==="
  "$BASE/.venv/bin/python" "$BASE/update_parcels.py"
  echo "=== done  $(date -Is) exit=$? ==="
} >> "$LOG" 2>&1

# crontab: one simple line, nothing to escape
30 2 * * * /srv/gis/run_parcels.sh

set -euo pipefail makes the wrapper fail loudly instead of continuing after a broken step, so cron sees a non-zero exit code.

Example 2: a preflight check the script runs on itself

import os, shutil, sys
from pathlib import Path

def preflight() -> None:
    problems = []

    base = Path(__file__).resolve().parent
    for p in (base / "data/raw", base / "data/out"):
        if not p.exists():
            problems.append(f"missing directory: {p}")

    try:
        import geopandas  # noqa: F401
    except ImportError as exc:
        problems.append(f"geopandas not importable by {sys.executable}: {exc}")

    if shutil.disk_usage(base).free < 2 * 1024**3:
        problems.append("less than 2 GB free on the output volume")

    for var in ("DB_URL",):
        if not os.environ.get(var):
            problems.append(f"environment variable {var} is not set")

    if problems:
        for p in problems:
            print(f"PREFLIGHT: {p}", file=sys.stderr)
        raise SystemExit(2)

preflight()

A job that refuses to start with a clear message beats one that half-runs at 02:30.

Example 3: logging that works in both contexts

import logging, sys
from pathlib import Path

LOG_DIR = Path(__file__).resolve().parent / "logs"
LOG_DIR.mkdir(exist_ok=True)

logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s %(levelname)-7s %(message)s",
    handlers=[
        logging.FileHandler(LOG_DIR / "parcels.log", encoding="utf-8"),
        logging.StreamHandler(sys.stdout),
    ],
)
log = logging.getLogger("parcels")
log.info("starting with interpreter %s", sys.executable)

The file handler means the run is recorded even when nobody captures stdout; the stream handler keeps interactive runs readable.

Example 4: the systemd equivalent, for jobs that matter

# /etc/systemd/system/parcels.service
[Service]
Type=oneshot
User=gis
WorkingDirectory=/srv/gis
EnvironmentFile=/srv/gis/job.env
ExecStart=/srv/gis/.venv/bin/python /srv/gis/update_parcels.py

# /etc/systemd/system/parcels.timer
[Timer]
OnCalendar=*-*-* 02:30:00
Persistent=true

[Install]
WantedBy=timers.target

WorkingDirectory, EnvironmentFile and journal logging are exactly the three things cron makes you build yourself — and Persistent=true runs a missed job after the machine was off.

Explanation

An interactive login does a great deal of invisible setup. The shell reads /etc/profile, ~/.profile, ~/.bashrc or their zsh equivalents; a conda or venv activation prepends a directory to PATH and exports library variables; the terminal supplies stdin, stdout and stderr; and your working directory is wherever you happen to be standing.

Layered anatomy of a cron wrapper script: shell options, cd, env file, interpreter, redirect. — A wrapper script re-creates each piece of the interactive environment, explicitly.

Cron does almost none of it. It starts your command with /bin/sh, a minimal PATH of roughly /usr/bin:/bin, the user's home as the working directory, and no profile. Every difference in behaviour follows from that. The wrong python is a PATH difference. The missing input file is a working-directory difference. The ProjError about a missing database is an environment-variable difference. The silence is the absence of a terminal.

This is also why "add it to the crontab and see" is such an expensive debugging loop. Each iteration costs a day. Making the environment explicit — absolute interpreter, absolute script, explicit working directory, explicit env file, explicit log — collapses the difference between the two contexts so that a manual run genuinely tests the scheduled one.

The final piece is observability. A scheduled job with no log is indistinguishable from a job that never ran, and after a few weeks nobody can say which. Logging start, finish, exit code and row counts turns the schedule into something you can audit, and gives an alerting rule something concrete to watch.

Edge cases or notes

Cron mails output by default: On a server with no MTA, that mail is discarded. Never rely on it; redirect explicitly.
PATH in a crontab is literal: Lines like PATH=$PATH:/opt/bin do not expand. Write the full value, or set PATH inside a wrapper script.
Overlapping runs: A job that takes longer than its interval will run concurrently with itself. Guard with flock -n /tmp/parcels.lock or a lock file in the script.
% needs escaping, # does not: Only percent has special meaning in the command field. A # starts a comment only at the beginning of a line.
Windows Task Scheduler has the same traps: Set "Start in" (the working directory), use the full path to python.exe in the venv's Scripts folder, and tick "Run whether user is logged on or not" — which also removes access to mapped network drives.
User crontabs differ from /etc/cron.d: System crontab files take an extra user field before the command. A missing user field is a common silent failure.
Time zones: Cron uses the system time zone, so a job scheduled for 02:30 may shift with daylight saving. systemd timers accept an explicit Timezone=.

Internal links

FAQ

Why does cron use a different Python than my terminal?

Because activating a virtualenv only modifies PATH in that shell session, and cron never reads your shell profile. Call the interpreter by absolute path — /srv/gis/.venv/bin/python — and no activation is needed.

Where does my scheduled script's output go if I do not redirect it?

Cron mails it to the crontab's owner. On most servers there is no mail transport configured, so the output is discarded. Always append to a log file with >> file 2>&1.

How can I test a cron job without waiting for the schedule?

Run the exact command under a stripped environment: env -i /bin/sh -c 'cd $HOME && /path/to/venv/bin/python /path/to/script.py'. That approximates cron closely enough to reproduce the usual failures.

Why does my script report a missing file that clearly exists?

The path is relative and cron's working directory is the user's home. Anchor paths to Path(__file__).resolve().parent, or add cd /srv/gis && to the crontab line.

What does a `%` in my crontab do?

It is converted to a newline, and everything after it becomes standard input for the command. Escape it as \%, or move the command into a wrapper script where the rule does not apply.

Should I use cron or systemd timers?

Systemd timers if the host has systemd: they give you a working directory, an environment file, journal logging, exit-code tracking, and Persistent=true for missed runs. Cron is fine for simple jobs behind a wrapper script.

How do I stop two runs overlapping?

Wrap the command in flock -n /var/lock/parcels.lock -c '…', which exits immediately if another run holds the lock. In Python, a lock file plus a check on the recorded process id works cross-platform.

Python GIS Script Works Manually but Not from Cron: How to Fix It #

Problem statement #

Quick answer #

What cron does not give you #

Step-by-step solution #

Capture the output before anything else #

Find out which interpreter cron is using #

Never rely on activate #

Set the working directory in the script #

Provide the environment variables the libraries need #

Escape percent signs #

Reproduce the failure on demand #

Code examples #

Example 1: a wrapper script that makes the job self-contained #

Example 2: a preflight check the script runs on itself #

Example 3: logging that works in both contexts #

Example 4: the systemd equivalent, for jobs that matter #

Explanation #

Edge cases or notes #

Internal links #

FAQ #

Why does cron use a different Python than my terminal? #

Where does my scheduled script's output go if I do not redirect it? #

How can I test a cron job without waiting for the schedule? #

Why does my script report a missing file that clearly exists? #

What does a % in my crontab do? #

Should I use cron or systemd timers? #

How do I stop two runs overlapping? #