The QGIS Processing Framework Explained

Problem statement

The Processing Toolbox holds well over a thousand entries, contributed by five or six different engines, and from Python they all reduce to one call:

processing.run("native:buffer", {"INPUT": layer, "DISTANCE": 25, "OUTPUT": "TEMPORARY_OUTPUT"})

That uniformity is not an accident, and understanding the machinery behind it answers most of the questions people have when they start automating QGIS:

why an algorithm id has a prefix, and why qgis: and native: both exist
why TEMPORARY_OUTPUT sometimes gives you a layer and sometimes a path
why the algorithm works in the GUI and reports "not found" in a script
how GRASS and SAGA algorithms appear alongside QGIS's own
what the Modeler is actually producing, and why it can be run headless

Processing is a plugin-based framework: a registry, a parameter system and a runner. Everything else — the Toolbox, the Modeler, qgis_process, batch mode — is a front end onto it.

Quick answer

Five concepts explain the whole framework:

Providers contribute algorithms and own an id prefix: native:, qgis:, gdal:, grass:, saga:, model:, script:
Algorithms declare typed parameters and typed outputs — that declaration is what generates the dialog, the CLI help and the validation
A context carries the project, CRS settings and temporary layer store; feedback carries progress and messages
processing.run() validates parameters, executes, and returns a results dictionary
The registry is empty until Processing.initialize() runs — which the GUI does for you and a script must do itself

from qgis.core import QgsApplication
import processing

registry = QgsApplication.processingRegistry()
print("providers :", [p.id() for p in registry.providers()])
print("algorithms:", len(registry.algorithms()))

alg = registry.algorithmById("native:buffer")
print(alg.displayName(), "→", [p.name() for p in alg.parameterDefinitions()])

Asking the registry beats guessing every time: it reflects your installation, not the documentation for some other version.

The shape of the framework

Layered stack from front ends through the registry, providers, algorithms and parameters. — Four front ends, one registry — which is why the Toolbox, the Modeler and the CLI agree.

Step-by-step solution

Vertical steps of a processing.run call: resolve id, validate parameters, prepare context, execute, return results. — What happens between your dictionary and the output layer.

Providers: where algorithms come from

from qgis.core import QgsApplication

for provider in QgsApplication.processingRegistry().providers():
    algorithms = provider.algorithms()
    print(f"{provider.id():12} {len(algorithms):>4} algorithms  {provider.name()}")

native        ~370 algorithms  QGIS (native c++)
qgis           ~90 algorithms  QGIS
gdal           ~60 algorithms  GDAL
3d              ~5 algorithms  QGIS (3D)
grass         ~300 algorithms  GRASS

Each provider is a plugin that registers its algorithms under a prefix. native: is the C++ implementation and is the fastest; qgis: is the older Python one, with some ids kept as aliases; gdal: wraps the GDAL command-line tools; grass: and saga: shell out to those packages and require them to be installed.

That model is why a missing algorithm is usually a missing provider — the algorithm exists, but nothing registered it in this process.

Algorithms declare their own interface

An algorithm is a class that describes its parameters and outputs. Everything else is generated from that description.

import processing

processing.algorithmHelp("native:buffer")

Buffer (native:buffer)
Arguments
---------
INPUT: Input layer        Argument type: source
DISTANCE: Distance        Argument type: distance
SEGMENTS: Segments        Argument type: number
END_CAP_STYLE: End cap style
        Argument type: enum
        Available values: 0: Round  1: Flat  2: Square
DISSOLVE: Dissolve result Argument type: boolean
OUTPUT: Buffered          Argument type: sink

The same declaration produces the Toolbox dialog, the batch-mode table, the qgis_process help output, and the validation that rejects a bad parameter before anything runs. Writing your own algorithm means writing that declaration — which is why a script algorithm gets a dialog for free.

Parameter types, and what they accept

import processing
from qgis.core import QgsVectorLayer

layer = QgsVectorLayer("data/parcels.gpkg|layername=parcels", "parcels", "ogr")

processing.run("native:buffer", {
    "INPUT": layer,                       # a source: layer object, path, or path|layername=
    "DISTANCE": 25,                       # a distance, in the layer's units
    "SEGMENTS": 8,                        # a number
    "END_CAP_STYLE": 0,                   # an enum — the integer, not the label
    "DISSOLVE": False,                    # a boolean
    "OUTPUT": "data/out/buffered.gpkg",   # a sink: a path, memory:, or TEMPORARY_OUTPUT
})

The types worth knowing: source (an input layer), sink (an output layer), enum (an integer), distance and number, field (a column name), extent (xmin,xmax,ymin,ymax[CRS]), crs, matrix, and multiple layers. A source accepts a QgsVectorLayer, a file path, or a URI string — which is what lets the same call work in a script and in the Modeler.

Outputs: files, memory, or the context

import processing
from qgis.core import QgsProcessingContext, QgsProcessingFeedback

context = QgsProcessingContext()          # keep this alive
feedback = QgsProcessingFeedback()

# a real file: survives the run
processing.run("native:buffer", {"INPUT": layer, "DISTANCE": 25,
                                 "OUTPUT": "data/out/buffered.gpkg"})

# temporary: lives in the context, ideal for chaining
result = processing.run("native:buffer",
                        {"INPUT": layer, "DISTANCE": 25, "OUTPUT": "TEMPORARY_OUTPUT"},
                        context=context, feedback=feedback)

out = context.getMapLayer(result["OUTPUT"])
print(type(out), out.featureCount() if out else "not in context")

TEMPORARY_OUTPUT is the reason chained algorithms are fast: intermediate results stay in memory in the context rather than being written and re-read. It is also the reason a result "disappears" if the context is garbage-collected — the layer belonged to it.

Context and feedback: the two objects to pass explicitly

from qgis.core import (QgsProcessingContext, QgsProcessingFeedback,
                       QgsProject, QgsCoordinateTransformContext)

context = QgsProcessingContext()
context.setProject(QgsProject.instance())         # so @project variables resolve
context.setInvalidGeometryCheck(
    QgsFeatureRequest.GeometryAbortOnInvalid)     # or SkipInvalid / NoCheck

class Verbose(QgsProcessingFeedback):
    def pushInfo(self, info):        print("INFO :", info)
    def pushWarning(self, warning):  print("WARN :", warning)
    def reportError(self, error, fatalError=False): print("ERROR:", error)
    def setProgress(self, progress): pass

result = processing.run("native:clip", params, context=context, feedback=Verbose())

Without a feedback object, the messages an algorithm emits — skipped features, invalid geometries, CRS notes — go nowhere. That is the usual reason an empty result seems to have no explanation.

The invalid-geometry policy

from qgis.core import QgsProcessingContext, QgsFeatureRequest

context = QgsProcessingContext()

# abort the whole run on the first invalid geometry
context.setInvalidGeometryCheck(QgsFeatureRequest.GeometryAbortOnInvalid)

# skip invalid features and carry on (the GUI default)
context.setInvalidGeometryCheck(QgsFeatureRequest.GeometrySkipInvalid)

# do not check at all — fastest, and you own the consequences
context.setInvalidGeometryCheck(QgsFeatureRequest.GeometryNoCheck)

This single setting explains a class of "the GUI and my script disagree" reports: the GUI skips invalid input features by default, so a script with a different policy can produce a different count from the same data.

Running algorithms from every front end

# Python API
import processing
processing.run("native:buffer", params)

# with the results loaded into the project (GUI only)
processing.runAndLoadResults("native:buffer", params)

# command line — same registry, same parameters
qgis_process run native:buffer -- INPUT=parcels.gpkg DISTANCE=25 OUTPUT=out.gpkg

# a saved model is just another algorithm
processing.run("model:parcel_cleanup", {"INPUT": "parcels.gpkg", "OUTPUT": "clean.gpkg"})

The uniformity is the payoff: anything the Toolbox can run, a script can run, the CLI can run, and a model can contain — because all four resolve the same id against the same registry.

Code examples

Example 1: explore the registry properly

from qgis.core import QgsApplication
import csv

def dump_algorithms(path="algorithms.csv"):
    registry = QgsApplication.processingRegistry()
    with open(path, "w", newline="", encoding="utf-8") as fh:
        writer = csv.writer(fh)
        writer.writerow(["id", "display_name", "provider", "group", "parameters", "outputs"])
        for alg in sorted(registry.algorithms(), key=lambda a: a.id()):
            writer.writerow([
                alg.id(), alg.displayName(), alg.provider().id(), alg.group(),
                "|".join(p.name() for p in alg.parameterDefinitions()),
                "|".join(o.name() for o in alg.outputDefinitions()),
            ])
    print(f"wrote {path}")

def find(term: str):
    registry = QgsApplication.processingRegistry()
    for alg in registry.algorithms():
        if term.lower() in alg.id().lower() or term.lower() in alg.displayName().lower():
            print(f"{alg.id():45} {alg.displayName()}")

find("dissolve")
dump_algorithms()

Example 2: inspect parameters programmatically

from qgis.core import (QgsApplication, QgsProcessingParameterEnum,
                       QgsProcessingParameterNumber, QgsProcessingParameterField)

def describe(alg_id: str) -> None:
    alg = QgsApplication.processingRegistry().algorithmById(alg_id)
    if alg is None:
        raise ValueError(f"{alg_id} is not registered")

    print(f"{alg.displayName()} ({alg.id()})")
    for param in alg.parameterDefinitions():
        line = f"  {param.name():22} {type(param).__name__.replace('QgsProcessingParameter', ''):14}"
        if isinstance(param, QgsProcessingParameterEnum):
            line += " options: " + ", ".join(f"{i}={v}" for i, v in enumerate(param.options()))
        elif isinstance(param, QgsProcessingParameterNumber):
            line += f" default={param.defaultValue()}"
        elif isinstance(param, QgsProcessingParameterField):
            line += f" of layer parameter {param.parentLayerParameterName()}"
        if param.flags() & param.FlagOptional:
            line += "  [optional]"
        print(line)

describe("native:joinattributesbylocation")

This is algorithmHelp() with structure, and it is what you want when generating configuration or validating a model programmatically.

Example 3: chain algorithms through one context

import processing
from qgis.core import QgsProcessingContext, QgsProcessingFeedback

def chain(source, steps, final_output):
    """Run a list of (alg_id, params) with temporary outputs between them."""
    context, feedback = QgsProcessingContext(), QgsProcessingFeedback()
    current = source

    for i, (alg_id, params) in enumerate(steps):
        is_last = i == len(steps) - 1
        run_params = {**params, "INPUT": current,
                      "OUTPUT": final_output if is_last else "TEMPORARY_OUTPUT"}
        result = processing.run(alg_id, run_params, context=context, feedback=feedback)
        current = result["OUTPUT"]
        print(f"{i+1}. {alg_id} → {'file' if is_last else 'memory'}")

    return current, context      # return the context so temporaries stay alive

output, context = chain(
    "data/raw/parcels.gpkg",
    [("native:fixgeometries", {}),
     ("native:reprojectlayer", {"TARGET_CRS": "EPSG:27700"}),
     ("native:buffer", {"DISTANCE": 25, "SEGMENTS": 8, "DISSOLVE": False}),
     ("native:dissolve", {"FIELD": ["class"]})],
    "data/out/parcels_zones.gpkg",
)

Returning the context alongside the result is the detail that prevents temporaries vanishing mid-chain.

Example 4: initialise Processing correctly in a script

import os, sys
os.environ.setdefault("QT_QPA_PLATFORM", "offscreen")

from qgis.core import QgsApplication

def boot(prefix="/usr", plugins="/usr/share/qgis/python/plugins"):
    QgsApplication.setPrefixPath(prefix, True)
    app = QgsApplication([], False)
    app.initQgis()                                   # 1. providers, CRS database

    if plugins not in sys.path:
        sys.path.append(plugins)
    from processing.core.Processing import Processing
    Processing.initialize()                          # 2. THEN the registry

    n = len(QgsApplication.processingRegistry().algorithms())
    if not n:
        raise RuntimeError("Processing registry is empty after initialize()")
    print(f"{n} algorithms available")
    return app

app = boot()
try:
    import processing
    print(processing.run("native:buffer", {
        "INPUT": "data/raw/parcels.gpkg", "DISTANCE": 25,
        "OUTPUT": "data/out/buffered.gpkg"})["OUTPUT"])
finally:
    from qgis.core import QgsProject
    QgsProject.instance().clear()
    app.exitQgis()

The order — initQgis() before Processing.initialize() — is the single most common mistake in headless PyQGIS, and it produces an empty registry rather than an error.

Explanation

Processing began life as the SEXTANTE plugin and was absorbed into QGIS as its general-purpose analysis framework. Its central design decision is that an algorithm declares its interface rather than implementing one, and everything else follows from that.

Grid of processing providers with their prefix, implementation, speed and requirements. — Six providers, one registry — and only two of them are always available.

Because parameters are typed objects with names, defaults, ranges and optionality, one description generates every front end. The Toolbox builds a dialog from it. Batch mode builds a table. qgis_process help prints it. The Modeler uses it to know which outputs can feed which inputs. And processing.run() uses it to validate your dictionary before anything executes — which is why a wrong parameter name fails immediately with a clear message rather than halfway through.

The provider model is what makes a thousand algorithms possible without QGIS implementing them. GRASS and SAGA providers shell out to those packages, translating parameters in and results back; the GDAL provider builds ogr2ogr and gdalwarp command lines; native: calls straight into C++. From the caller's side they are identical, which is a genuinely impressive abstraction and also the reason availability varies between installations: no GRASS on the machine, no grass: algorithms in the registry.

Context and feedback are the two objects that carry everything an algorithm needs that is not a parameter. The context holds the project, the transform context, the invalid-geometry policy and — importantly — a temporary layer store, which is what TEMPORARY_OUTPUT writes into. Feedback carries progress and messages back out. Passing your own instances of both is what turns a script from a black box into something you can watch and control.

Finally, the registry's emptiness at start-up explains the most common headless failure. Inside QGIS, the application and Processing were initialised long before your code ran. In a standalone script, QgsApplication.initQgis() sets up providers and the CRS database, and only then can Processing.initialize() populate the algorithm registry. Reverse the order and you get a registry with nothing in it — and the error you eventually see is "algorithm not found", pointing at the wrong thing entirely.

Edge cases or notes

native: beats qgis: where both exist: The C++ implementations are faster; some qgis: ids survive only as aliases.
Enum parameters take integers: The labels in the dialog are display text. algorithmHelp() lists the mapping.
TEMPORARY_OUTPUT needs a living context: Keep a reference, or fetch the layer before the context goes out of scope.
runAndLoadResults is GUI-only: In a headless script there is no canvas to load into.
Invalid-geometry policy differs by front end: The GUI skips invalid features by default; set the policy explicitly in scripts for reproducible counts.
Provider availability is per installation: Check registry.providers() rather than assuming GRASS or SAGA exist.
Models and scripts are algorithms too: They register under model: and script: once the profile is loaded, which is why a headless run may need the .model3 file path instead.

Internal links

FAQ

What is the difference between `native:` and `qgis:` algorithms?

native: are the C++ implementations and are faster. qgis: are the older Python ones; many were ported and their ids kept as aliases. Prefer native: and confirm against your own registry.

Why does my script say the algorithm is not found?

Almost always because Processing.initialize() has not run, or ran before QgsApplication.initQgis(). The registry is empty until both have happened in that order.

What does `TEMPORARY_OUTPUT` actually create?

A layer in the QgsProcessingContext, held in memory. Retrieve it with context.getMapLayer(result["OUTPUT"]), and keep the context alive or the layer goes with it.

How do I find an algorithm's parameter names?

processing.algorithmHelp("id"), qgis_process help id, or iterate alg.parameterDefinitions() for structured access. The Processing History panel also records the exact call the GUI made.

Why do GRASS or SAGA algorithms not appear?

Their providers are separate plugins that need those packages installed and registered. In a headless environment you usually have to add the provider explicitly after Processing.initialize().

What is the context for?

It carries the project, the coordinate transform context, the invalid-geometry policy and the temporary layer store. Passing your own gives you control over all four and keeps temporary outputs alive.

Can I write my own algorithm?

Yes — subclass QgsProcessingAlgorithm, declare parameters and outputs, and implement processAlgorithm. It then appears in the Toolbox, in batch mode, in the Modeler and in qgis_process with no extra work.

The QGIS Processing Framework Explained #

Problem statement #

Quick answer #

The shape of the framework #

Step-by-step solution #

Providers: where algorithms come from #

Algorithms declare their own interface #

Parameter types, and what they accept #

Outputs: files, memory, or the context #

Context and feedback: the two objects to pass explicitly #

The invalid-geometry policy #

Running algorithms from every front end #

Code examples #

Example 1: explore the registry properly #

Example 2: inspect parameters programmatically #

Example 3: chain algorithms through one context #

Example 4: initialise Processing correctly in a script #

Explanation #

Edge cases or notes #

Internal links #

FAQ #

What is the difference between native: and qgis: algorithms? #

Why does my script say the algorithm is not found? #

What does TEMPORARY_OUTPUT actually create? #

How do I find an algorithm's parameter names? #

Why do GRASS or SAGA algorithms not appear? #

What is the context for? #

Can I write my own algorithm? #