Configuration vs Code: What Belongs in a Config File

Problem statement

Two failure modes, and most projects manage both at once. First, everything hard-coded:

gdf = gpd.read_file("/home/anna/data/parcels_2026_08.shp")
gdf = gdf[gdf["area"] > 500]
gdf.to_crs("EPSG:27700").to_file("/home/anna/out/result.gpkg")

Running last month means editing the script. Running on the server means editing it again. Two people running it means two divergent copies.

Then the overcorrection:

steps:
  - if: "{{ params.mode == 'full' }}"
    then:
      - foreach: "{{ layers }}"
        do:
          op: clip
          args: {boundary: "{{ item.boundary }}", keep: "{{ item.keep | default(true) }}"}

The logic has escaped into YAML, where it cannot be tested, debugged or type-checked, and where a typo fails at 02:30 instead of at review time.

The useful question is not "config or code?" but "which of these two things is this?" — and there is a clean answer.

Quick answer

Configuration is what varies between runs without changing what the job does:

Config: paths, CRS, thresholds, date ranges, credentials' names, feature flags, output formats
Code: logic, control flow, conditionals, loops, ordering, anything with an if
Layer the sources: defaults in code → config file → environment → command line
Validate the resolved config at start-up, and fail with a clear message
Record what was actually used in the run metadata, not just the file you meant to use

from dataclasses import dataclass
from pathlib import Path
import os, yaml

@dataclass(frozen=True)
class Config:
    input: Path
    output: Path
    crs: str = "EPSG:27700"
    min_area_m2: float = 5.0
    overwrite: bool = False

    @classmethod
    def load(cls, path: Path, **overrides) -> "Config":
        data = yaml.safe_load(Path(path).read_text(encoding="utf-8")) or {}
        for key in ("input", "output"):
            if key in data:
                data[key] = (Path(path).parent / data[key]).resolve()
        env = {k[4:].lower(): v for k, v in os.environ.items() if k.startswith("GIS_")}
        merged = {**data, **env, **{k: v for k, v in overrides.items() if v is not None}}
        return cls(**{k: v for k, v in merged.items() if k in cls.__annotations__})

config = Config.load("configs/monthly.yml", min_area_m2=10)
print(config)

A frozen dataclass gives you defaults, types, autocompletion and one place where every value is resolved — which is most of what people actually want from "configuration management".

Which side of the line?

Grid classifying paths, thresholds, logic, secrets and schedules as config or code. — The test is whether changing it changes *what the job does* or only *which data it does it to*.

Step-by-step solution

Layered configuration precedence from code defaults through file, environment to command line. — Four layers, each overriding the one below — and the resolved result is what you record.

The dividing line

# ── configuration: varies per run, does not change the logic ──────────────
INPUT = "data/raw/parcels.gpkg"
TARGET_CRS = "EPSG:27700"
MIN_AREA_M2 = 5
DROP_CLASSES = ["exempt", "unknown"]
NOTIFY_ON_FAILURE = True

# ── code: the logic itself ────────────────────────────────────────────────
def clean(gdf, target_crs, min_area_m2, drop_classes):
    out = gdf[~gdf["class"].isin(drop_classes)].copy()
    out = out.to_crs(target_crs)
    out["area_m2"] = out.area
    return out[out["area_m2"] >= min_area_m2]

The signal that something belongs in config: two people would run it with different values, or the same person would next month. The signal that something belongs in code: it has a condition, a loop, or an ordering.

Layer the sources, with a clear precedence

import argparse, os, yaml
from pathlib import Path

DEFAULTS = {"crs": "EPSG:27700", "min_area_m2": 5.0, "overwrite": False}

def resolve_config(argv=None) -> dict:
    ap = argparse.ArgumentParser()
    ap.add_argument("--config", type=Path, default=Path("configs/default.yml"))
    ap.add_argument("--input", type=Path)
    ap.add_argument("--crs")
    ap.add_argument("--min-area-m2", type=float, dest="min_area_m2")
    ap.add_argument("--overwrite", action="store_true", default=None)
    args = ap.parse_args(argv)

    config = dict(DEFAULTS)                                     # 1. code defaults

    if args.config.exists():                                    # 2. config file
        config.update(yaml.safe_load(args.config.read_text()) or {})

    for key in list(config):                                    # 3. environment
        env_value = os.environ.get(f"GIS_{key.upper()}")
        if env_value is not None:
            config[key] = type(config[key])(env_value) if not isinstance(config[key], bool) \
                else env_value.lower() in {"1", "true", "yes"}

    for key, value in vars(args).items():                       # 4. command line
        if key != "config" and value is not None:
            config[key] = value

    config["_source"] = str(args.config.resolve())
    return config

Four layers, in the order everyone expects: code defaults are the fallback, the file is the project's settings, the environment is the deployment's, and the command line is this run's. Recording _source matters — "which config file did that run actually use?" is a question that comes up often.

Validate the resolved values, not the file

from pathlib import Path
import pyproj

def validate(config: dict) -> dict:
    problems = []

    for key in ("input", "output"):
        if key not in config:
            problems.append(f"missing required key: {key}")

    if "input" in config and not Path(config["input"]).exists():
        problems.append(f"input does not exist: {Path(config['input']).resolve()}")

    try:
        pyproj.CRS.from_user_input(config["crs"])
    except Exception as exc:
        problems.append(f"invalid crs {config['crs']!r}: {exc}")

    if not 0 <= float(config["min_area_m2"]) < 1e9:
        problems.append(f"min_area_m2 out of range: {config['min_area_m2']}")

    unknown = set(config) - set(DEFAULTS) - {"input", "output", "_source"}
    problems += [f"unknown key (typo?): {k}" for k in sorted(unknown)]

    if problems:
        raise ValueError("configuration is invalid:\n  " + "\n  ".join(problems))
    return config

Flagging unknown keys is as valuable as checking required ones: min_area_m instead of min_area_m2 otherwise leaves the default silently in place, and the run produces plausible, wrong output.

Keep logic out of the config

# ✗ this is a programming language now
steps:
  - name: clip
    when: "region != 'north' and mode == 'full'"
    retry_if: "error contains 'timeout'"

# ✓ configuration describes *what*, code decides *how*
region: north
mode: full
clip_boundary: data/ref/city.gpkg
retries: 3

# the conditional lives in code, where it can be read and tested
if config["mode"] == "full" and config["region"] != "north":
    gdf = clip_to(gdf, boundary)

The tell that logic has leaked into configuration: the file contains if, when, unless, expressions in strings, or templating. At that point you have an untested, untyped language with no debugger — the worst of both worlds.

Environments: same code, different values

configs/
  base.yml         # everything shared
  dev.yml          # local paths, small samples, verbose logging
  prod.yml         # real paths, full data, alerting on

def load_environment(name: str, folder=Path("configs")) -> dict:
    base = yaml.safe_load((folder / "base.yml").read_text()) or {}
    layer = yaml.safe_load((folder / f"{name}.yml").read_text()) or {}
    merged = {**base, **layer}
    merged["_environment"] = name
    return merged

config = load_environment(os.environ.get("GIS_ENV", "dev"))

Splitting by environment rather than copying the whole file keeps the shared settings in one place, so a change to the CRS does not have to be made three times.

Secrets are configuration that must not be in the file

import os

# ✗ never
# db_password: "Summer2026!"

# ✓ the config names the secret; the value comes from the environment
config = {"db_user": "gis", "db_host": "db.internal", "db_password_env": "PGPASSWORD"}
password = os.environ[config["db_password_env"]]

A config file lives in version control; a credential must not. Naming the environment variable in the config keeps the wiring visible without putting the value anywhere it can leak.

Record what was used

import json
from datetime import datetime, timezone
from pathlib import Path

def record_config(config: dict, dest=Path("logs/runs")) -> Path:
    safe = {k: str(v) for k, v in config.items() if "password" not in k and "secret" not in k}
    dest.mkdir(parents=True, exist_ok=True)
    run_id = datetime.now(timezone.utc).strftime("%Y%m%dT%H%M%SZ")
    path = dest / f"{run_id}-config.json"
    path.write_text(json.dumps(safe, indent=2), encoding="utf-8")
    return path

The resolved configuration — after every layer has been applied — is what determines the output. Recording it makes an old result explainable; recording only the file path does not, because the environment and command line are invisible.

Code examples

Example 1: a typed configuration with validation

from dataclasses import dataclass, field, asdict
from pathlib import Path
from typing import Literal
import os, yaml

@dataclass(frozen=True)
class PipelineConfig:
    input: Path
    output: Path
    boundary: Path | None = None
    crs: str = "EPSG:27700"
    min_area_m2: float = 5.0
    drop_classes: tuple[str, ...] = ()
    overwrite: bool = False
    log_level: Literal["DEBUG", "INFO", "WARNING", "ERROR"] = "INFO"

    def __post_init__(self):
        import pyproj
        if not self.input.exists():
            raise FileNotFoundError(f"input not found: {self.input}")
        pyproj.CRS.from_user_input(self.crs)          # raises on nonsense
        if self.min_area_m2 < 0:
            raise ValueError("min_area_m2 must be >= 0")

    @classmethod
    def from_file(cls, path: Path, **overrides) -> "PipelineConfig":
        path = Path(path).resolve()
        raw = yaml.safe_load(path.read_text(encoding="utf-8")) or {}

        for key in ("input", "output", "boundary"):
            if raw.get(key):
                raw[key] = (path.parent / raw[key]).resolve()
        if "drop_classes" in raw:
            raw["drop_classes"] = tuple(raw["drop_classes"])

        known = set(cls.__dataclass_fields__)
        unknown = set(raw) - known
        if unknown:
            raise ValueError(f"unknown config keys: {sorted(unknown)}")

        env = {}
        for name in known:
            value = os.environ.get(f"GIS_{name.upper()}")
            if value is not None:
                env[name] = value

        merged = {**raw, **env, **{k: v for k, v in overrides.items() if v is not None}}
        merged["_source"] = str(path)
        merged.pop("_source")
        return cls(**merged)

    def redacted(self) -> dict:
        return {k: str(v) for k, v in asdict(self).items()}

config = PipelineConfig.from_file("configs/monthly.yml", min_area_m2=10)
print(config.redacted())

For anything larger, pydantic gives the same shape with coercion, ranges and readable error reports for far less code.

Example 2: a config file that reads like documentation

# configs/monthly.yml
# Monthly parcel refresh. Paths are relative to this file.

input: ../data/raw/parcels.gpkg
boundary: ../data/ref/city_boundary.gpkg
output: ../data/out/parcels_clean.gpkg

# British National Grid — outputs must match the council's GIS
crs: "EPSG:27700"

# Parcels below this are digitising slivers, not real plots
min_area_m2: 5

# Exempt and unknown parcels are excluded from the published layer
drop_classes: ["exempt", "unknown"]

overwrite: false
log_level: INFO

Comments explaining why a value is what it is are the most useful thing in a config file and the first thing people leave out.

Example 3: show the resolved configuration on start-up

import logging

def log_config(config, log=logging.getLogger("pipeline")) -> None:
    log.info("configuration:")
    for key, value in sorted(vars(config).items() if hasattr(config, "__dict__")
                             else config.items()):
        if any(s in key.lower() for s in ("password", "secret", "token")):
            value = "***"
        log.info("  %-16s %s", key, value)

Three lines that answer "what did it actually run with?" every single time, in the log where the failure also is.

Example 4: test the resolution order

import pytest

def test_command_line_beats_environment(tmp_path, monkeypatch):
    config_file = tmp_path / "c.yml"
    config_file.write_text("crs: EPSG:4326\nmin_area_m2: 1\n")
    monkeypatch.setenv("GIS_MIN_AREA_M2", "20")

    config = resolve_config(["--config", str(config_file), "--min-area-m2", "99"])
    assert config["min_area_m2"] == 99          # CLI wins
    assert config["crs"] == "EPSG:4326"         # file wins over the code default

def test_unknown_key_is_rejected(tmp_path):
    config_file = tmp_path / "c.yml"
    config_file.write_text("min_area_m: 5\n")   # typo
    with pytest.raises(ValueError, match="unknown"):
        PipelineConfig.from_file(config_file)

Explanation

The purpose of configuration is to make one program serve many situations without being edited. That is a substitution mechanism — replacing values — and it works beautifully as long as only values vary.

Triage table of configuration smells and what each one indicates. — Six smells that mean a boundary has been drawn in the wrong place.

Logic is different in kind. A conditional expresses a decision, and decisions need the things code has: types, tests, a debugger, version control with meaningful diffs, and review. Encoding them in YAML gets you none of that. The result is a system where the most consequential parts — when a step is skipped, which branch a run takes — are the least examinable, and where a typo produces a wrong result rather than a crash.

The layering convention exists because different values belong to different owners. Defaults belong to the program. The config file belongs to the project and lives in version control. Environment variables belong to the deployment: the server knows where the data root is and which database to talk to, and that should not require editing a committed file. Command-line flags belong to this single run. Precedence flows from most general to most specific, which is why every tool that does this — from git to docker to pytest — orders it the same way.

Validation belongs at the boundary, applied to the resolved configuration rather than the file. By then the environment and the command line have had their say, and what remains is exactly the set of values the run will use. Checking those values, including rejecting keys you do not recognise, converts a whole class of silent misconfiguration into an immediate, specific error — the difference between a run that buffers by the default 0 metres for a fortnight and one that refuses to start.

The last piece is recording. Because the resolved configuration is assembled from four sources, the file alone does not explain a run. Writing the resolved values (with secrets redacted) into the run record makes an old output explainable months later, and it is three lines of code.

Edge cases or notes

Paths in config are relative to the config file: That is what an editor expects. Resolve them at load time.
Environment variables are strings: "false" is truthy. Convert explicitly, especially booleans.
YAML type surprises: no becomes False, 4326 an int, 1.10 the float 1.1. Quote anything that must stay text.
Do not template the config: Jinja in YAML is a language in disguise. If you need computation, do it in code.
Secrets never go in the file: Name the environment variable in config; keep the value in the environment or a secret store.
Defaults belong in code, not in the file: Then a missing key is safe, and the file only records deliberate deviations.
A config that no one reads is a smell: If nobody can say what a setting does, it probably encodes a decision that belongs in code.

Internal links

FAQ

What belongs in a config file?

Values that vary between runs without changing the job's behaviour: paths, CRS, thresholds, date ranges, output formats, feature flags, and the names of secrets.

What should stay in code?

Anything with a decision in it — conditionals, loops, step ordering, error handling. If the YAML contains if or a templated expression, the logic has escaped.

What order should configuration sources be applied in?

Code defaults, then the config file, then environment variables, then command-line flags. Most general to most specific, which is the convention every familiar tool follows.

Where do secrets go?

In the environment or a secret store, never in the file. The config can name the environment variable so the wiring is documented without the value being exposed.

Why validate unknown keys?

Because a typo like min_area_m leaves the default in place and the run produces plausible, wrong output. Rejecting unrecognised keys turns that into an immediate error.

YAML, TOML or JSON?

YAML for human-edited files — it supports comments, which matter more than people expect. TOML is a good alternative. JSON has no comments and is best kept for machine-generated records.

Do I need pydantic or dynaconf?

Not for a handful of settings; a frozen dataclass with a from_file classmethod covers it. Reach for a library when you have many settings, nested structures or several environments.

Configuration vs Code: What Belongs in a Config File #

Problem statement #

Quick answer #

Which side of the line? #

Step-by-step solution #

The dividing line #

Layer the sources, with a clear precedence #

Validate the resolved values, not the file #

Keep logic out of the config #

Environments: same code, different values #

Secrets are configuration that must not be in the file #

Record what was used #

Code examples #

Example 1: a typed configuration with validation #

Example 2: a config file that reads like documentation #

Example 3: show the resolved configuration on start-up #

Example 4: test the resolution order #

Explanation #

Edge cases or notes #

Internal links #

FAQ #

What belongs in a config file? #

What should stay in code? #

What order should configuration sources be applied in? #

Where do secrets go? #

Why validate unknown keys? #

YAML, TOML or JSON? #

Do I need pydantic or dynaconf? #