Python GIS Script Works Manually but Not from Cron: How to Fix It

On this page

Problem statement

The script runs perfectly from your terminal. Scheduled, it produces nothing — no output file, no log line, and often no error anywhere you thought to look.

$ python update_parcels.py
wrote data/out/parcels_2026-09-29.gpkg (2513 parcels) in 0.2s

# crontab entry
30 2 * * * python update_parcels.py
# … next morning: no new file, no message

Cron did run it. What it ran was a different environment: a different PATH, a different working directory, no virtualenv, no shell profile, and no terminal to print to. Anything your script inherited from your interactive session is simply absent.

What is missing, in the order it usually bites:

python on cron's PATH is the system interpreter, not your virtualenv — or, on current Debian and Ubuntu, does not exist at all
the working directory is the user's home, so every relative path misses
environment variables set in .bashrc or .profile are never loaded
output goes to the void because cron mails stdout and no mailer is configured
the crontab line has an unescaped %, which cron treats as a newline
the file is not executable, or the shebang points at the wrong interpreter
GDAL/PROJ environment variables (PROJ_LIB, GDAL_DATA) are unset

The same list applies almost unchanged to Windows Task Scheduler, systemd timers, and Airflow workers. The difference is never "cron is broken" — it is that an interactive shell does a lot of setup you never see.

Quick answer

To make a scheduled GIS script behave like the manual run:

use absolute paths for the interpreter, the script, and every data path
redirect stdout and stderr to a log file so failures leave a trace
set the working directory explicitly in the script, not in the crontab
escape % as \% in crontab lines
run the exact cron command by hand with env -i to reproduce the failure

# crontab -e
30 2 * * * cd /srv/gis && /srv/gis/.venv/bin/python /srv/gis/update_parcels.py >> /srv/gis/logs/parcels.log 2>&1

Three absolute paths and one redirect fix the large majority of cases. .venv/bin/python runs the script with the virtualenv's packages without any activation step — that is what a venv's interpreter does.

What cron does not give you

Four differences account for nearly every "works manually, fails scheduled" report.

Capture output, then fix interpreter, directory and environment

Debug by making the failure visible first — guessing at causes wastes days of overnight runs.

Capture the output before anything else

A scheduled job that fails invisibly cannot be debugged. Redirect both streams to a file, appending so you keep the history.

30 2 * * * /srv/gis/.venv/bin/python /srv/gis/update_parcels.py >> /srv/gis/logs/parcels.log 2>&1

2>&1 must come after the >>, or stderr still goes to cron's mail. Add a timestamp per run so a log covering weeks stays readable:

30 2 * * * { date -Is; /srv/gis/.venv/bin/python /srv/gis/update_parcels.py; echo "exit=$?"; } >> /srv/gis/logs/parcels.log 2>&1

On systemd, output goes to the journal automatically — journalctl -u parcels.service. On Windows Task Scheduler, the task history pane records the exit code, and you still want a redirect for the text.

Find out which interpreter cron is using

The single most common failure is the wrong Python. Have the script report its own environment on start-up.

import sys, os
from pathlib import Path

print("python  :", sys.executable)
print("version :", sys.version.split()[0])
print("cwd     :", Path.cwd())
print("user    :", os.environ.get("USER") or os.environ.get("USERNAME"))
print("PATH    :", os.environ.get("PATH"))

Run it manually, then run it scheduled, and compare the two logs. If sys.executable is /usr/bin/python3 under cron and /srv/gis/.venv/bin/python3 in your terminal, you have found the problem — and the fix is to call the venv interpreter by absolute path.

Never rely on `activate`

Activating a virtualenv is a shell convenience; it only edits PATH. Cron does not run your shell profile, so there is nothing to activate into. Two reliable options:

# option 1 — call the venv interpreter directly (preferred)
/srv/gis/.venv/bin/python /srv/gis/update_parcels.py

# option 2 — run a login shell that sources the profile, then activate
0 2 * * * /bin/bash -lc 'source /srv/gis/.venv/bin/activate && python /srv/gis/update_parcels.py'

Option 1 has fewer moving parts, and it keeps working when the profile changes. Use option 2 only when the script genuinely needs variables that live in the profile.

Set the working directory in the script

Relative paths resolve against the process working directory, which cron sets to the user's home. Anchoring inside the script means the job behaves identically no matter who starts it or from where.

from pathlib import Path
import os

BASE = Path(__file__).resolve().parent
os.chdir(BASE)                    # optional, but makes relative paths predictable

SRC = BASE / "data" / "raw"
OUT = BASE / "data" / "out"
OUT.mkdir(parents=True, exist_ok=True)

Alternatively cd /srv/gis && … in the crontab line. Doing it in the script is more robust, because it also covers manual runs from an unexpected directory.

Provide the environment variables the libraries need

GDAL, PROJ and some cloud drivers read configuration from the environment. Conda installations in particular set PROJ_LIB and GDAL_DATA during activation.

import os
os.environ.setdefault("PROJ_LIB", "/srv/gis/.venv/share/proj")
os.environ.setdefault("GDAL_DATA", "/srv/gis/.venv/share/gdal")

Check before copying those paths. With GDAL and PROJ installed from pip wheels, as in the test above, each carries its own data and needs neither variable — /srv/gis/.venv/share does not even exist, and the job ran without them. They matter for system or conda builds of GDAL; pyproj.datadir.get_data_dir() and gdal.GetConfigOption("GDAL_DATA") tell you where your installation looks.

Better: keep the job's variables in one file and load it, so the same values are used by every entry point.

# /srv/gis/job.env
PROJ_LIB=/srv/gis/.venv/share/proj
GDAL_DATA=/srv/gis/.venv/share/gdal
DB_URL=postgresql://gis:secret@db.internal/parcels

30 2 * * * set -a && . /srv/gis/job.env && set +a && /srv/gis/.venv/bin/python /srv/gis/update_parcels.py >> /srv/gis/logs/parcels.log 2>&1

Cron also supports plain KEY=value lines at the top of a crontab, but those are not expanded like shell variables — PATH=$PATH:/usr/local/bin does not work there.

Escape percent signs

In a crontab, an unescaped % becomes a newline, and everything after it is fed to the command as standard input. Any date format string will hit this.

# broken: cron truncates at the first %
0 2 * * * /srv/gis/run.sh --date $(date +%Y-%m-%d)

# correct
0 2 * * * /srv/gis/run.sh --date $(date +\%Y-\%m-\%d)

This is a good reason to keep crontab lines trivial and put the logic in a wrapper script, where no escaping is needed.

Reproduce the failure on demand

You do not have to wait until 02:30 to test. Strip the environment and run the exact command:

env -i PATH=/usr/bin:/bin HOME="$HOME" SHELL=/bin/sh /bin/sh -c 'cd $HOME && /srv/gis/.venv/bin/python /srv/gis/update_parcels.py'

env -i starts with an empty environment, and the variables after it put back exactly what cron sets. Give it the PATH explicitly: with none at all, Debian's /bin/sh falls back to its own default, /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin, which finds interpreters in /usr/local/bin that cron's /usr/bin:/bin does not. If it fails here, you can iterate in seconds instead of nights.

What a real cron daemon did with each line

The redirect goes first because without it the failure leaves no trace at all.

What cron actually did with each line

Every failure above was reproduced with a real cron daemon: Debian 13 in a container, a user gis, a virtualenv at /srv/gis/.venv with GeoPandas installed from pip, and a nightly job, update_parcels.py, that reads 2,513 of Manchester's Land Registry parcels from the relative path data/raw/parcels.gpkg, adds an area column and writes a dated copy. These lines went into the user's crontab, all set to run every minute:

* * * * * python update_parcels.py
* * * * * python update_parcels.py >> /srv/gis/logs/naive.log 2>&1
* * * * * python3 /srv/gis/update_parcels.py >> /srv/gis/logs/system_python3.log 2>&1
* * * * * /srv/gis/.venv/bin/python /srv/gis/update_parcels.py >> /srv/gis/logs/no_cd.log 2>&1
* * * * * cd /srv/gis && /srv/gis/.venv/bin/python /srv/gis/update_parcels.py >> /srv/gis/logs/parcels.log 2>&1
* * * * * echo "stamp $(date +%Y-%m-%d)" >> /srv/gis/logs/percent_raw.log 2>&1
* * * * * echo "stamp $(date +\%Y-\%m-\%d)" >> /srv/gis/logs/percent_escaped.log 2>&1
* * * * * env | sort > /srv/gis/logs/cron_env.txt

A minute later:

cron_env.txt          HOME=/home/gis  LOGNAME=gis  PATH=/usr/bin:/bin  PWD=/home/gis  SHELL=/bin/sh
naive.log             /bin/sh: 1: python: not found
system_python3.log    ModuleNotFoundError: No module named 'geopandas'
no_cd.log             pyogrio.errors.DataSourceError: data/raw/parcels.gpkg: No such file or directory
parcels.log           wrote data/out/parcels_2026-09-29.gpkg (2513 parcels) in 0.2s
percent_escaped.log   stamp 2026-09-29
percent_raw.log       (never created)

The first line — the one from the problem statement — left nothing anywhere: no output file, and its error went to a mailer the container does not have. With a redirect, the same line says why: there is no python on cron's PATH. python3 exists but is the system interpreter, which has never heard of GeoPandas. The virtualenv's interpreter gets further and then misses the data, because cron starts in the home directory. And the unescaped % did not produce a wrong date — it ended the command at +, so the redirect that would have created the log never ran either. Only the line with an absolute interpreter and a cd worked.

Example 1: a wrapper script that makes the job self-contained

#!/usr/bin/env bash
# /srv/gis/run_parcels.sh
set -uo pipefail                 # not -e: a failing job must still reach the "done" line

BASE="/srv/gis"
LOG="$BASE/logs/parcels-$(date +%Y%m%d).log"

cd "$BASE"
set -a; . "$BASE/job.env"; set +a

{
  echo "=== start $(date -Is) ==="
  "$BASE/.venv/bin/python" "$BASE/update_parcels.py"
  status=$?                      # capture it before anything else runs
  echo "=== done  $(date -Is) exit=$status ==="
} >> "$LOG" 2>&1
exit "$status"

The first version of this wrapper had set -euo pipefail and logged exit=$? after $(date -Is). Both hide a failure, and a job that exits with status 3 shows it: -e ends the wrapper as soon as Python fails, so the "done" line is never written, and without -e the $? is read after the date substitution and reports exit=0. Capture the status on the line after the job.

# crontab: one simple line, nothing to escape
30 2 * * * /srv/gis/run_parcels.sh

set -euo pipefail makes the wrapper fail loudly instead of continuing after a broken step, so cron sees a non-zero exit code.

Example 2: a preflight check the script runs on itself

import os, shutil, sys
from pathlib import Path

def preflight() -> None:
    problems = []

    base = Path(__file__).resolve().parent
    for p in (base / "data/raw", base / "data/out"):
        if not p.exists():
            problems.append(f"missing directory: {p}")

    try:
        import geopandas  # noqa: F401
    except ImportError as exc:
        problems.append(f"geopandas not importable by {sys.executable}: {exc}")

    if shutil.disk_usage(base).free < 2 * 1024**3:
        problems.append("less than 2 GB free on the output volume")

    for var in ("DB_URL",):
        if not os.environ.get(var):
            problems.append(f"environment variable {var} is not set")

    if problems:
        for p in problems:
            print(f"PREFLIGHT: {p}", file=sys.stderr)
        raise SystemExit(2)

preflight()

A job that refuses to start with a clear message beats one that half-runs at 02:30.

Example 3: logging that works in both contexts

import logging, sys
from pathlib import Path

LOG_DIR = Path(__file__).resolve().parent / "logs"
LOG_DIR.mkdir(exist_ok=True)

logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s %(levelname)-7s %(message)s",
    handlers=[
        logging.FileHandler(LOG_DIR / "parcels.log", encoding="utf-8"),
        logging.StreamHandler(sys.stdout),
    ],
)
log = logging.getLogger("parcels")
log.info("starting with interpreter %s", sys.executable)

The file handler means the run is recorded even when nobody captures stdout; the stream handler keeps interactive runs readable.

Example 4: the systemd equivalent, for jobs that matter

# /etc/systemd/system/parcels.service
[Service]
Type=oneshot
User=gis
WorkingDirectory=/srv/gis
EnvironmentFile=/srv/gis/job.env
ExecStart=/srv/gis/.venv/bin/python /srv/gis/update_parcels.py

# /etc/systemd/system/parcels.timer
[Timer]
OnCalendar=*-*-* 02:30:00
Persistent=true

[Install]
WantedBy=timers.target

WorkingDirectory, EnvironmentFile and journal logging are exactly the three things cron makes you build yourself — and Persistent=true runs a missed job after the machine was off.

What an interactive login sets up for you

An interactive login does a great deal of invisible setup. The shell reads /etc/profile, ~/.profile, ~/.bashrc or their zsh equivalents; a conda or venv activation prepends a directory to PATH and exports library variables; the terminal supplies stdin, stdout and stderr; and your working directory is wherever you happen to be standing.

A wrapper script re-creates each piece of the interactive environment, explicitly.

Cron does almost none of it. It starts your command with /bin/sh, a minimal PATH of roughly /usr/bin:/bin, the user's home as the working directory, and no profile. Every difference in behaviour follows from that. The wrong python is a PATH difference. The missing input file is a working-directory difference. The ProjError about a missing database is an environment-variable difference. The silence is the absence of a terminal.

This is also why "add it to the crontab and see" is such an expensive debugging loop. Each iteration costs a day. Making the environment explicit — absolute interpreter, absolute script, explicit working directory, explicit env file, explicit log — collapses the difference between the two contexts so that a manual run genuinely tests the scheduled one.

The final piece is observability. A scheduled job with no log is indistinguishable from a job that never ran, and after a few weeks nobody can say which. Logging start, finish, exit code and row counts turns the schedule into something you can audit, and gives an alerting rule something concrete to watch.

Lost cron mail and a minimal PATH

Cron mails output by default: On a server with no MTA, that mail is discarded. Never rely on it; redirect explicitly.
PATH in a crontab is literal: Lines like PATH=$PATH:/opt/bin do not expand. Write the full value, or set PATH inside a wrapper script.
Overlapping runs: A job that takes longer than its interval will run concurrently with itself. Guard with flock -n /tmp/parcels.lock or a lock file in the script.
% needs escaping, # does not: Only percent has special meaning in the command field. A # starts a comment only at the beginning of a line.
Windows Task Scheduler has the same traps: Set "Start in" (the working directory), use the full path to python.exe in the venv's Scripts folder, and tick "Run whether user is logged on or not" — which also removes access to mapped network drives.
User crontabs differ from /etc/cron.d: System crontab files take an extra user field before the command. A missing user field is a common silent failure.
Time zones: Cron uses the system time zone, so a job scheduled for 02:30 may shift with daylight saving. systemd timers accept an explicit Timezone=.

Internal links

FAQ

Why does cron use a different Python than my terminal?

Because activating a virtualenv only modifies PATH in that shell session, and cron never reads your shell profile. Call the interpreter by absolute path — /srv/gis/.venv/bin/python — and no activation is needed.

Where does my scheduled script's output go if I do not redirect it?

Cron mails it to the crontab's owner. On most servers there is no mail transport configured, so the output is discarded. Always append to a log file with >> file 2>&1.

How can I test a cron job without waiting for the schedule?

Run the exact command under cron's environment: env -i PATH=/usr/bin:/bin HOME="$HOME" /bin/sh -c 'cd $HOME && /path/to/venv/bin/python /path/to/script.py'. Set PATH yourself — an empty one makes the shell fall back to a broader default than cron uses.

Why does my script report a missing file that clearly exists?

The path is relative and cron's working directory is the user's home. Anchor paths to Path(__file__).resolve().parent, or add cd /srv/gis && to the crontab line.

What does a `%` in my crontab do?

It is converted to a newline, and everything after it becomes standard input for the command. Escape it as \%, or move the command into a wrapper script where the rule does not apply.

Should I use cron or systemd timers?

Systemd timers if the host has systemd: they give you a working directory, an environment file, journal logging, exit-code tracking, and Persistent=true for missed runs. Cron is fine for simple jobs behind a wrapper script.

How do I stop two runs overlapping?

Wrap the command in flock -n /var/lock/parcels.lock -c '…', which exits immediately if another run holds the lock. In Python, a lock file plus a check on the recorded process id works cross-platform.