⚙️ How-To QGIS & PyQGIS Batch processing QGIS Automation · step 9 Notebook By · Updated

How to Batch Process Layers in QGIS with PyQGIS

On this page

QGIS has a batch mode built into every algorithm dialog, and it is genuinely useful right up to the moment you need a condition, a second step, or a report of what failed. Then you want a loop. The loop is not hard — for path in folder.glob("*.gpkg") around a processing.run call is most of it — but the difference between a script that survives a folder of real data and one that dies on file seven comes down to three habits: isolate each file's failure, mirror the input structure deliberately, and write a report at the end.

Problem statement

You have a folder of layers, or one GeoPackage with twenty of them, and one operation to apply to each. The obstacles:

  • The built-in batch dialog runs one algorithm. Your workflow is reproject, then fix geometries, then clip — three dialogs, three passes, intermediate files everywhere.
  • One bad file kills the run. File 7 of 200 has a null geometry, the exception propagates, and files 8–200 never ran.
  • No record of what happened. The run finished; you do not know which files were skipped, which produced zero features, or which took ten minutes.
  • Output naming is ad hoc. Results land in one flat folder with names that collide, or lose the subfolder structure that meant something.
  • Layers inside containers are invisible. A GeoPackage holding twelve layers looks like one file to a glob.
  • It is slow. Sequential, single-process, and the machine has eight cores idle.

This page ends with a folder in, a mirrored folder out, a CSV report, and a non-zero exit code if anything failed.

Quick answer

Wrap the algorithm in a loop and put the try inside the loop, not around it.

The five stages of a PyQGIS batch loop. the loop body is one file’s whole lifeDiscoverrglob + sublayersLoadQgsVectorLayerRunprocessing.runWriteone GeoPackageRecorda row per fileDiscover, load, run, write, record — and the last one is not optional spatialworkflow.io
Discover, load, run, write, record — the loop body is one file's whole life.
import csv
from pathlib import Path
import processing
from qgis.core import QgsVectorLayer

SRC, DST = Path("data/raw"), Path("data/processed")
DST.mkdir(parents=True, exist_ok=True)
rows = []

for path in sorted(SRC.glob("*.gpkg")):
    try:
        layer = QgsVectorLayer(str(path), path.stem, "ogr")
        if not layer.isValid():
            raise RuntimeError("layer failed to load")

        out = DST / f"{path.stem}_buffer.gpkg"
        processing.run("native:buffer", {
            "INPUT": layer, "DISTANCE": 25, "SEGMENTS": 8,
            "DISSOLVE": False, "OUTPUT": str(out),
        })
        rows.append({"file": path.name, "status": "ok", "features": layer.featureCount()})
    except Exception as exc:
        rows.append({"file": path.name, "status": "failed", "error": str(exc)})

with open(DST / "report.csv", "w", newline="") as fh:
    w = csv.DictWriter(fh, fieldnames=["file", "status", "features", "error"])
    w.writeheader()
    w.writerows(rows)

for r in rows:
    print(f"{r['file']:20} {r['status']:7} {r.get('features', '')}{r.get('error', '')[:60]}")
broken.gpkg          failed  layer failed to load
cardiff_parks.gpkg   ok      149
city.gpkg            ok      3495
parks.gpkg           ok      275
rail.gpkg            ok      847
roads.gpkg           ok      3495

The 404 page fails and is reported. Look at city.gpkg, though: 3,495 features, the same as roads.gpkg. Opening a GeoPackage by its path gives you its first layer, so the container's railways, parks and schools were never buffered, and nothing says so. Enumerate the layers inside containers — step 1.

That runs headlessly with the bootstrap from running PyQGIS headless, and the algorithm call is the one from running Processing algorithms from Python.

Discover inputs, isolate failures, skip finished work

Discover the inputs, including the ones inside containers

A folder of shapefiles is easy. A folder of GeoPackages is not, because each one may hold several layers.

from pathlib import Path
from osgeo import ogr

def discover(folder: Path):
    """Yield (source_uri, label) for every vector layer under `folder`."""
    for path in sorted(folder.rglob("*")):
        if path.suffix.lower() in {".shp", ".geojson", ".fgb"}:
            yield str(path), path.stem
        elif path.suffix.lower() in {".gpkg", ".sqlite"}:
            try:
                ds = ogr.Open(str(path))           # GDAL 3.12 raises rather than returning None
            except RuntimeError:
                ds = None
            if ds is None:
                yield str(path), path.stem         # let processing fail it, and the report record it
                continue
            for i in range(ds.GetLayerCount()):
                name = ds.GetLayerByIndex(i).GetName()
                yield f"{path}|layername={name}", f"{path.stem}__{name}"
            ds = None

Two details earn their place. rglob walks subfolders, which is what you want once the archive grows — the mirroring pattern from processing files in nested subfolders applies unchanged. And the label combines container and layer name, so two layers called roads in different GeoPackages do not overwrite each other.

If you would rather not add a GDAL import, QGIS can enumerate sublayers itself:

from qgis.core import Qgis, QgsProviderRegistry

parts = QgsProviderRegistry.instance().querySublayers("data/raw/city.gpkg",
                                                      Qgis.SublayerQueryFlag.CountFeatures)
for p in parts:
    print(p.name(), p.uri(), p.featureCount())
roads data/raw/city.gpkg|layername=roads 3495
rail data/raw/city.gpkg|layername=rail 847
parks data/raw/city.gpkg|layername=parks 275
schools data/raw/city.gpkg|layername=schools 243
sources data/raw/city.gpkg|layername=sources 2

Without Qgis.SublayerQueryFlag.CountFeatures every count comes back as -2 — QGIS's code for "not counted yet" — because counting means reading each layer, and the plain query only lists them.

Put the failure boundary around one file

A try around the loop versus a try inside the loop. where the try sitstry around the loop✕file 7 raises✕files 8–200 never run✕one traceback, no report✕no record of what worked✕a rerun starts overtry inside the loop✓file 7 records failed✓files 8–200 still run✓199 results and 1 error✓the report names the file✓a rerun skips the doneOne level of indentation decides what a bad file costs spatialworkflow.io
Where the try sits decides whether a bad file costs you one row or the whole night.

The rule is simple and constantly broken: one try per item, inside the loop. A try around the loop turns a single malformed geometry into a total loss. Inside the loop, that file records failed, the loop continues, and you have 199 results plus one clear error to investigate.

Catch broadly here — except Exception — because the failure modes are genuinely varied: a QgsProcessingException from the algorithm, an OSError from a locked file, a RuntimeError you raised for an invalid layer. What matters is that the loop survives and the reason is recorded.

Make the output path a pure function of the input path

Naming is where batch jobs quietly corrupt themselves. Derive the destination mechanically so re-running produces the same layout:

def output_for(src: Path, root: Path, dst_root: Path, suffix: str) -> Path:
    rel = src.relative_to(root).with_suffix("")
    out = dst_root / rel.parent / f"{rel.name}_{suffix}.gpkg"
    out.parent.mkdir(parents=True, exist_ok=True)
    return out

data/raw/north/parcels.shp becomes data/processed/north/parcels_buffer.gpkg. The subfolder survives, the extension is normalised to GeoPackage, and nothing can collide unless the inputs already did.

Skip work that is already done

Any batch you run more than once should be idempotent and cheap on the second pass:

if out.exists() and out.stat().st_mtime > src.stat().st_mtime:
    rows.append({"file": src.name, "status": "skipped"})
    continue

That is the smallest possible version of the caching idea: if the output is newer than the input, there is nothing to do. For a run that may be interrupted, the fuller treatment in building a resumable batch job records completion in a manifest instead of trusting timestamps.

Chain several algorithms per file

The reason to write a loop instead of using the batch dialog is that the loop body can be a whole workflow:

def process_one(uri: str, out_path: Path) -> dict:
    import processing                            # after QGIS has started
    from qgis.core import QgsVectorLayer

    layer = QgsVectorLayer(uri, "in", "ogr")
    if not layer.isValid():
        raise RuntimeError(f"invalid layer: {uri}")

    stats = {"features_in": layer.featureCount(), "crs_in": layer.crs().authid()}

    reprojected = processing.run("native:reprojectlayer", {
        "INPUT": layer, "TARGET_CRS": "EPSG:27700", "OUTPUT": "TEMPORARY_OUTPUT",
    })["OUTPUT"]

    fixed = processing.run("native:fixgeometries", {
        "INPUT": reprojected, "OUTPUT": "TEMPORARY_OUTPUT",
    })["OUTPUT"]

    result = processing.run("native:clip", {
        "INPUT": fixed, "OVERLAY": "data/district.geojson", "OUTPUT": str(out_path),
    })["OUTPUT"]

    written = QgsVectorLayer(str(out_path), "out", "ogr")
    stats["features_out"] = written.featureCount()
    return stats

Three algorithms, one file written, and the counts in and out returned so the report can flag layers that lost everything to the clip.

Report on every run

import csv

FIELDS = ["label", "status", "features_in", "features_out", "crs_in", "seconds", "error"]

def write_report(rows, path):
    with open(path, "w", newline="") as fh:
        w = csv.DictWriter(fh, fieldnames=FIELDS, extrasaction="ignore")
        w.writeheader()
        w.writerows(rows)

    failed = sum(1 for r in rows if r["status"] == "failed")
    empty = sum(1 for r in rows if r.get("features_out") == 0)
    print(f"{len(rows)} layers · {failed} failed · {empty} produced zero features")
    return failed

Return the failure count and use it as the exit code, so a scheduler notices. The empty-output count is the check that catches the silent disaster — every file "succeeded", every output has zero features, because the clip boundary was in the wrong CRS.

Batching Bristol layers with QGIS's own Python

The data used on this page

Real layers for Bristol, built with nothing but QGIS's own Python and GDAL. The district is the City of Bristol from the ONS Local Authority Districts (May 2024) boundaries, as data/district.geojson in the British National Grid. data/raw/ holds OpenStreetMap extracts (© OpenStreetMap contributors, ODbL) as GeoPackages — Bristol's parks in the British National Grid, its major roads and railways in WGS84, Cardiff's parks, which lie outside the district, and broken.gpkg, a web server's 404 page saved under a GeoPackage name — plus city.gpkg, one container holding roads, railways, parks and schools as layers alongside a plain attribute table. The code ran on QGIS 4.2 in headless mode; qgis_headless.py, written here, is the small context manager the scripts import. The OpenStreetMap layers are copies of Overpass's answers saved on 10 October 2026.

import csv
import json
import os
from pathlib import Path
import zipfile

import requests
from osgeo import gdal

gdal.UseExceptions()
os.environ.setdefault("QT_QPA_PLATFORM", "offscreen")

Path("qgis_headless.py").write_text('''from contextlib import contextmanager

@contextmanager
def qgis_headless():
    """Start QGIS without a GUI, initialise Processing, and shut down afterwards."""
    from qgis.core import QgsApplication
    app = QgsApplication([], False)
    app.initQgis()
    from processing.core.Processing import Processing
    Processing.initialize()
    try:
        yield app
    finally:
        app.exitQgis()
''')

BBOX = "51.39,-2.73,51.55,-2.50"
QUERIES = {"roads": f'way["highway"~"^(motorway|trunk|primary|secondary)$"]({BBOX});out geom;',
           "rail": f'way["railway"="rail"]({BBOX});out geom;',
           "parks": f'way["leisure"="park"]({BBOX});out geom;',
           "schools": f'nwr["amenity"="school"]({BBOX});out center;',
           "cardiff_parks": 'way["leisure"="park"](51.44,-3.25,51.53,-3.12);out geom;'}

def overpass_geojson(name):
    cache = Path(f"osm_{name}.json")
    if not cache.exists():
        # live: requests.post("https://overpass-api.de/api/interpreter", data={"data": "[out:json][timeout:120];" + QUERIES[name]},
        #                     timeout=180, headers={"User-Agent": "spatialworkflow.io guide"})
        # read instead as these queries answered on 10 October 2026, so the layers stay put
        if not Path("osm-bristol-layers.zip").exists():
            reply = requests.get("https://data.spatialworkflow.io/osm-bristol-layers-2026-10-10.zip", timeout=120)
            reply.raise_for_status()
            Path("osm-bristol-layers.zip").write_bytes(reply.content)
        zipfile.ZipFile("osm-bristol-layers.zip").extract(cache.name)
    features = []
    for e in json.loads(cache.read_text())["elements"]:
        if "geometry" in e:
            xy = [[p["lon"], p["lat"]] for p in e["geometry"]]
            geom = ({"type": "Polygon", "coordinates": [xy]} if "parks" in name and xy[0] == xy[-1]
                    else {"type": "LineString", "coordinates": xy})   # a closed road is a roundabout, not an area
        else:
            c = e.get("center", e)
            geom = {"type": "Point", "coordinates": [c["lon"], c["lat"]]}
        features.append({"type": "Feature", "geometry": geom,
                         "properties": {"osm_id": e["id"], "name": e.get("tags", {}).get("name")}})
    path = Path(f"osm_{name}.geojson")
    path.write_text(json.dumps({"type": "FeatureCollection", "features": features}))
    return str(path)

raw = Path("data/raw")
raw.mkdir(parents=True, exist_ok=True)
if not Path("data/district.geojson").exists():
    Path("data/district.geojson").write_bytes(requests.get(
        "https://services1.arcgis.com/ESMARspQHYMw9BZ9/arcgis/rest/services/"
        "Local_Authority_Districts_May_2024_Boundaries_UK_BFC/FeatureServer/0/query"
        "?where=LAD24CD%3D%27E06000023%27&outFields=LAD24CD,LAD24NM&outSR=27700&f=geojson", timeout=120).content)
    gdal.VectorTranslate(str(raw / "parks.gpkg"), overpass_geojson("parks"), dstSRS="EPSG:27700", layerName="parks")
    gdal.VectorTranslate(str(raw / "roads.gpkg"), overpass_geojson("roads"), layerName="roads")
    gdal.VectorTranslate(str(raw / "rail.gpkg"), overpass_geojson("rail"), layerName="rail")
    gdal.VectorTranslate(str(raw / "cardiff_parks.gpkg"), overpass_geojson("cardiff_parks"), layerName="parks")
    (raw / "broken.gpkg").write_text("<html><body><h1>404 Not Found</h1></body></html>")
    for name in ("roads", "rail", "parks", "schools"):
        gdal.VectorTranslate("data/raw/city.gpkg", overpass_geojson(name), layerName=name,
                             accessMode="update" if Path("data/raw/city.gpkg").exists() else None)
    Path("sources.csv").write_text("layer,source,licence\nroads,OpenStreetMap,ODbL\nparks,OpenStreetMap,ODbL\n")
    gdal.VectorTranslate("data/raw/city.gpkg", "sources.csv", layerName="sources", accessMode="update")
Path("config").mkdir(exist_ok=True)
Path("config/buffers.csv").write_text("layer,distance_m\nroads,15\nrail,40\n")

# QGIS once for the whole notebook, as the Python console would have it
from qgis.core import QgsApplication
qgs = QgsApplication([], False)
qgs.initQgis()
from processing.core.Processing import Processing
Processing.initialize()
from qgis.core import Qgis
print("QGIS", Qgis.QGIS_VERSION, "|", sorted(p.name for p in raw.iterdir()))
QGIS 4.2.2-Belém do Pará | ['broken.gpkg', 'cardiff_parks.gpkg', 'city.gpkg', 'parks.gpkg', 'rail.gpkg', 'roads.gpkg']

Example 1: The complete batch script

%%writefile batch_qgis.py
#!/usr/bin/env python3
"""Batch: reproject → fix → clip every vector layer under data/raw."""
import csv
import logging
import sys
import time
from pathlib import Path

from osgeo import ogr
from qgis_headless import qgis_headless      # the context manager written in the setup

SRC = Path("data/raw")
DST = Path("data/processed")
log = logging.getLogger("batch")

def discover(folder: Path):
    """Yield (source_uri, label) for every vector layer under `folder`."""
    for path in sorted(folder.rglob("*")):
        if path.suffix.lower() in {".shp", ".geojson", ".fgb"}:
            yield str(path), path.stem
        elif path.suffix.lower() in {".gpkg", ".sqlite"}:
            try:
                ds = ogr.Open(str(path))           # GDAL 3.12 raises rather than returning None
            except RuntimeError:
                ds = None
            if ds is None:
                yield str(path), path.stem         # let processing fail it, and the report record it
                continue
            for i in range(ds.GetLayerCount()):
                name = ds.GetLayerByIndex(i).GetName()
                yield f"{path}|layername={name}", f"{path.stem}__{name}"
            ds = None

def process_one(uri: str, out_path: Path) -> dict:
    import processing                            # after QGIS has started
    from qgis.core import QgsVectorLayer

    layer = QgsVectorLayer(uri, "in", "ogr")
    if not layer.isValid():
        raise RuntimeError(f"invalid layer: {uri}")

    stats = {"features_in": layer.featureCount(), "crs_in": layer.crs().authid()}

    reprojected = processing.run("native:reprojectlayer", {
        "INPUT": layer, "TARGET_CRS": "EPSG:27700", "OUTPUT": "TEMPORARY_OUTPUT",
    })["OUTPUT"]

    fixed = processing.run("native:fixgeometries", {
        "INPUT": reprojected, "OUTPUT": "TEMPORARY_OUTPUT",
    })["OUTPUT"]

    result = processing.run("native:clip", {
        "INPUT": fixed, "OVERLAY": "data/district.geojson", "OUTPUT": str(out_path),
    })["OUTPUT"]

    written = QgsVectorLayer(str(out_path), "out", "ogr")
    stats["features_out"] = written.featureCount()
    return stats

FIELDS = ["label", "status", "features_in", "features_out", "crs_in", "seconds", "error"]

def write_report(rows, path):
    with open(path, "w", newline="") as fh:
        w = csv.DictWriter(fh, fieldnames=FIELDS, extrasaction="ignore")
        w.writeheader()
        w.writerows(rows)

    failed = sum(1 for r in rows if r["status"] == "failed")
    empty = sum(1 for r in rows if r.get("features_out") == 0)
    print(f"{len(rows)} layers · {failed} failed · {empty} produced zero features")
    return failed

def main():
    import processing
    from qgis.core import QgsVectorLayer

    rows = []
    for uri, label in discover(SRC):
        started = time.perf_counter()
        row = {"label": label, "status": "ok"}
        out = DST / f"{label}.gpkg"
        out.parent.mkdir(parents=True, exist_ok=True)
        try:
            row.update(process_one(uri, out))
        except Exception as exc:
            row["status"] = "failed"
            row["error"] = f"{type(exc).__name__}: {exc}"
            log.warning("✗ %s — %s", label, exc)
        else:
            log.info("✓ %s (%d → %d features)", label, row["features_in"], row["features_out"])
        row["seconds"] = round(time.perf_counter() - started, 2)
        rows.append(row)

    return write_report(rows, DST / "report.csv")

if __name__ == "__main__":
    logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)-7s %(message)s")
    DST.mkdir(parents=True, exist_ok=True)
    with qgis_headless():
        failures = main()
    sys.exit(1 if failures else 0)
import sys
!{sys.executable} batch_qgis.py 2>&1 | sed 's/^[0-9-]* [0-9:,]* //'
print(open("data/processed/report.csv").read())
WARNING ✗ broken — invalid layer: data/raw/broken.gpkg
INFO    ✓ cardiff_parks__parks (149 → 0 features)
INFO    ✓ city__roads (3495 → 2405 features)
INFO    ✓ city__rail (847 → 561 features)
INFO    ✓ city__parks (275 → 221 features)
INFO    ✓ city__schools (243 → 182 features)
INFO    ✓ city__sources (2 → 0 features)
INFO    ✓ parks__parks (275 → 221 features)
INFO    ✓ rail__rail (847 → 561 features)
INFO    ✓ roads__roads (3495 → 2405 features)
10 layers · 1 failed · 2 produced zero features
label,status,features_in,features_out,crs_in,seconds,error
broken,failed,,,,0.0,RuntimeError: invalid layer: data/raw/broken.gpkg
cardiff_parks__parks,ok,149,0,EPSG:4326,0.1,
city__roads,ok,3495,2405,EPSG:4326,0.07,
city__rail,ok,847,561,EPSG:4326,0.04,
city__parks,ok,275,221,EPSG:4326,0.03,
city__schools,ok,243,182,EPSG:4326,0.02,
city__sources,ok,2,0,,0.02,
parks__parks,ok,275,221,EPSG:27700,0.03,
rail__rail,ok,847,561,EPSG:4326,0.04,
roads__roads,ok,3495,2405,EPSG:4326,0.07,

Example 2: Batching over layers inside one GeoPackage

Often the "folder" is a single container, and the loop is over sublayers.

from qgis.core import QgsProviderRegistry
import processing

SRC = "data/raw/city.gpkg"
Path("out").mkdir(exist_ok=True)

from qgis.core import Qgis

for sub in QgsProviderRegistry.instance().querySublayers(SRC):
    if sub.wkbType() == Qgis.WkbType.NoGeometry:          # skip non-spatial tables
        print("skipped", sub.name(), "(no geometry)")
        continue
    processing.run("native:buffer", {
        "INPUT": sub.uri(),
        "DISTANCE": 10, "SEGMENTS": 8, "DISSOLVE": False,
        "OUTPUT": f"ogr:dbname='out/city_buffers.gpkg' table=\"{sub.name()}\" (geom)",
    })
print([p.name() for p in QgsProviderRegistry.instance().querySublayers("out/city_buffers.gpkg")])
skipped sources (no geometry)
['roads', 'rail', 'parks', 'schools']

The first version of this loop skipped layers whose wkbType() was 0. That is Unknown — what QGIS reports for a layer whose geometry type it has not resolved, such as one holding more than one type — not "no geometry": the attribute table here reports 100, NoGeometry, so the old test let it through to the buffer algorithm and could skip a real layer instead.

Writing every result back into a single output GeoPackage keeps the deliverable to one file, which is usually what the person receiving it wants.

What three ways of choosing layers did with a GeoPackage holding four layers and a table. city.gpkg: roads, rail, parks, schools + a tablelayers processedthe tableby path✕roads only (3,495)ignoredsublayers, wkbType 0✓all four✕processed toosublayers, NoGeometry✓all four✓skippedwkbType 0 is Unknown; a table with no geometry reports 100, NoGeometry spatialworkflow.io
Only enumerating the sublayers and testing for NoGeometry handles the container correctly.

Example 3: Per-file settings from a CSV

Not every layer wants the same parameters. Drive them from a table rather than an if ladder.

import csv
from pathlib import Path
import processing

settings = {}
with open("config/buffers.csv") as fh:
    for row in csv.DictReader(fh):
        settings[row["layer"]] = float(row["distance_m"])

DEFAULT = 25.0
for path in sorted(Path("data/raw").glob("*.gpkg")):
    distance = settings.get(path.stem, DEFAULT)
    try:
        processing.run("native:buffer", {
            "INPUT": str(path), "DISTANCE": distance, "SEGMENTS": 8,
            "DISSOLVE": False, "OUTPUT": f"data/processed/{path.stem}_buffer.gpkg",
        })
        print(f"{path.stem:14} buffered {distance:g} m")
    except Exception as exc:
        print(f"{path.stem:14} failed: {str(exc).splitlines()[-1]}")
broken         failed: Could not load source layer for INPUT: data/raw/broken.gpkg not found
cardiff_parks  buffered 25 m
city           buffered 25 m
parks          buffered 25 m
rail           buffered 40 m
roads          buffered 15 m

The CSV is reviewable, diffable, and editable by someone who does not write Python — the same argument that makes a YAML config worth the effort at pipeline scale.

Example 4: Parallelising across processes

QGIS objects are not thread-safe, so parallelism means processes, each with its own application.

%%writefile parallel_qgis.py
from concurrent.futures import ProcessPoolExecutor
from pathlib import Path

from batch_qgis import discover, process_one, write_report

def start_qgis():
    """Runs once in each worker: start QGIS and keep it for every layer that worker gets."""
    global QGS
    from qgis.core import QgsApplication
    QGS = QgsApplication([], False)
    QGS.initQgis()
    from processing.core.Processing import Processing
    Processing.initialize()

def worker(uri_label):
    uri, label = uri_label
    try:
        stats = process_one(uri, Path("data/processed") / f"{label}.gpkg")
        return {"label": label, "status": "ok", **stats}
    except Exception as exc:
        return {"label": label, "status": "failed", "error": str(exc)}

if __name__ == "__main__":
    Path("data/processed").mkdir(parents=True, exist_ok=True)
    items = list(discover(Path("data/raw")))
    with ProcessPoolExecutor(max_workers=4, initializer=start_qgis) as pool:
        rows = list(pool.map(worker, items))
    write_report(rows, Path("data/processed/report.csv"))
!{sys.executable} -W ignore parallel_qgis.py 2>&1 | tail -1
10 layers · 1 failed · 2 produced zero features

QGIS starts once per worker, in the pool's initializer, and stays up for every layer that worker is given. The first version of this example started and stopped QGIS inside each task instead; a worker that has called exitQgis() cannot start QGIS again, so the second task in any worker crashed the process, and the pool failed with BrokenProcessPool after writing one file. The workers also import their function from a module, because under Python 3.14's forkserver start method they cannot use one defined in a notebook. The trade-offs are the same as in speeding up batch GIS jobs with parallel processing.

Example 5: Using qgis_process with no Python at all

For a genuinely single-algorithm batch, the shell is enough:

for f in data/raw/*.gpkg; do
  qgis_process run native:buffer -- \
    INPUT="$f" DISTANCE=25 SEGMENTS=8 \
    OUTPUT="data/processed/$(basename "${f%.gpkg}")_buffer.gpkg" \
    || echo "FAILED: $f" >> data/processed/failures.txt
done

On this folder it writes five buffers and one FAILED: data/raw/broken.gpkg line — and, like the quick answer, takes only the first layer from city.gpkg.

The || is the shell's version of per-item error isolation. This is a perfectly good answer for a one-off; move to Python when you need a second step, a condition, or a real report.

Finally, shut QGIS down when the notebook is done with it — a process that exits with QGIS still initialised can crash on the way out:

qgs.exitQgis()

The GIS and the accounting

The complete script’s report for ten Bristol layers: reproject, fix and clip to the district. the report catches what exceptions do notlayerstatusinoutcity__roadsok3,4952,405city__railok847561city__schoolsok243182cardiff_parks__parksok1490city__sourcesok20brokenfailed——zero out, nothing raisedCounts in and out, side by side, on every run spatialworkflow.io
The zero-out row is the one the report exists to catch — it never raised an exception.

A batch script has two jobs, and only one of them is the GIS. The first is applying the operation; the second is accounting — knowing what was attempted, what succeeded, what changed, and what to look at. Scripts that skip the accounting feel finished and are not, because the failure mode they cannot see is the one that matters: a run where every file succeeded and every output is wrong.

That is why the report carries counts, not just statuses. A clip whose overlay is in the wrong CRS produces empty outputs and raises nothing at all — no exception, no warning, just twelve valid GeoPackages containing zero features. features_in and features_out side by side make it obvious in one glance. The same logic applies to a reprojection that silently assumed a CRS, or a join that matched nothing; the assertion you want is not "did it run" but "is the result the shape I expected", which is the argument behind validating a GeoDataFrame against a schema and applies equally here.

Choosing between a Python loop, the built-in batch dialog, and not using QGIS at all. the batch dialog is genuinely good, within its limitsWhat does the run need?one algorithmthe batch dialogno code at alla chain, or a skip rulea Python loopand a run reportno QGIS algorithmGeoPandaswalking folders is not GISReach for PyQGIS because the work needs a QGIS algorithm, not because the files are spatial spatialworkflow.io
Much of what a batch script does — walking folders, naming outputs — is not GIS at all.

The loop-versus-batch-dialog choice is worth stating plainly, because the dialog is genuinely good. Use the built-in batch mode when the job is one algorithm, run once, with no conditions. Write a loop when you need more than one step, per-file parameters, error isolation, a report, resumability, or a schedule — that is, whenever it will run again. The moment the words "and then also" appear in the description, you want the loop.

The final structural point is about where QGIS fits at all. Much of what a batch script does — walking folders, building output paths, handling errors, writing CSV — is plain Python, identical to the general batch processing pattern used with GeoPandas. QGIS supplies the middle of the loop: the algorithms. That means you can develop and test the skeleton without QGIS running, and it means the choice between PyQGIS and GeoPandas for a given job is really a choice about which algorithms you need, not about which framework batches better. PyQGIS vs GeoPandas works through that decision.

Mixed geometry, truncated names and locked files

Mixed geometry types across a folder

A folder of "the same" layers often is not: some point, some multipoint, some with Z values. Algorithms that write a sink declare one geometry type, so a mixed batch can fail on file 40 with a type mismatch. Either branch on layer.wkbType(), or normalise first with native:promotetomulti and native:dropmzvalues.

Shapefile field-name truncation

Writing results to shapefile silently truncates field names to ten characters and can collide two columns into one. Write GeoPackage unless something downstream demands otherwise — the details are in shapefile column names truncated.

Locked outputs on Windows

A GeoPackage still open in a running QGIS Desktop cannot be overwritten by a script. The error is an unhelpful write failure. Close the layer in the desktop, or write to a temporary name and rename after — a rename is atomic and cheap.

Empty inputs are not errors

A layer with zero features is valid and most algorithms will happily produce an empty output. Decide explicitly whether that is a skipped or a failed in your report; the worst answer is ok.

Memory over a long run

One process working through a thousand layers accumulates provider caches. If resident memory climbs steadily, chunk the batch across process restarts rather than hunting for the leak — and see fixing memory errors with large files for the general shape of the problem.

Sorting matters for reproducibility

glob returns filesystem order, which differs between machines. Always sorted() the discovery so two runs process the same files in the same sequence, and a diff of two reports lines up.

FAQ

Why write a loop when QGIS has a batch mode in every dialog?

Because the dialog runs exactly one algorithm with one set of parameters and reports nothing you can act on programmatically. A loop gives you multi-step workflows, per-file parameters, error isolation, a CSV report, resumability, and the ability to schedule the whole thing. Use the dialog for a one-off; write a loop for anything that will run twice.

How do I process every layer inside a GeoPackage?

Enumerate the sublayers rather than treating the file as one dataset. QgsProviderRegistry.instance().querySublayers(path) returns a part for each layer with its full URI, name, and geometry type; iterate those and pass sub.uri() as the algorithm input. The GDAL route — ogr.Open(path) and GetLayerByIndex — works equally well.

Where should the try/except go?

Inside the loop, wrapping the work for one file. Around the loop it converts a single bad file into total failure, which is the most common and most expensive mistake in batch GIS code. Catch Exception broadly, record the file and the error text in your report row, and let the loop continue.

How do I make the batch resumable after an interruption?

Skip any item whose output already exists and is newer than its input, or — more robustly — record completed items in a manifest file and consult it at startup. The manifest survives partially written outputs, which timestamps do not. The full pattern is in building a resumable batch GIS job.

Can I run PyQGIS batches in parallel threads?

No. QGIS objects and the Processing framework are not thread-safe, and threading them produces crashes rather than speedups. Parallelise with processes instead — ProcessPoolExecutor, one QgsApplication per worker — and chunk the work so the startup cost is amortised over several layers per process.