🧭 Concept Metadata, licensing & provenance Data cleaning Metadata, Licensing & Provenance · step 1 of 20 Updated

Spatial metadata explained: what a dataset must tell you

Problem statement

Somebody hands you boundaries_final_v3.gpkg. It opens, it draws, the geometry is valid. You still cannot use it, because none of the questions that decide whether it is fit for the job have answers: what do these polygons represent, when were they true, where did they come from, how accurate are they, and are you allowed to publish a map made from them.

Metadata is the record that answers those questions, and it is not documentation. Documentation is prose somebody may read; metadata is structured fields a program can check, a catalogue can index and a pipeline can refuse to run without. The distinction matters because the failure mode is always the same — the person who knew is gone, and the file is still in production.

This guide sets out the fields that earn their place, where each one can live, and which of them your file formats will actually carry.

Quick answer

Nine fields answer almost every question anyone asks of a spatial dataset:

metadata = {
    "title":        "Coastal flood risk zones, East Sussex",
    "abstract":     "Modelled 1-in-200-year extent at 2 m resolution, ...",
    "extent":       {"bbox": [-0.42, 50.72, 0.38, 51.02], "crs": "EPSG:4326"},
    "temporal":     {"content_date": "2025-11-01", "valid_until": "2027-11-01"},
    "lineage":      "EA LiDAR DTM 2024 → pysheds D8 → 200-year return period",
    "accuracy":     {"positional_m": 2.0, "vertical_m": 0.15},
    "licence":      "OGL-UK-3.0",
    "attribution":  "Contains Environment Agency data © Crown copyright",
    "contact":      "[email protected]",
}

Everything else is elaboration. If a record has these nine and they are true, a stranger can decide whether to use the dataset; if it has forty fields and no lineage, they cannot.

Checklist of the nine metadata fields that decide whether a dataset can be used.
Nine fields, four questions: what is it, when was it true, where did it come from, what may I do with it.

Step-by-step solution

1. Separate discovery from use

Metadata has two audiences. A discovery record — title, abstract, extent, keywords, licence — exists so somebody can find the dataset and decide whether to open it. A use record — CRS, schema, units, accuracy, nodata, lineage — exists so somebody who has opened it does not misinterpret it. Catalogues want the first; your pipeline needs the second. Most published metadata has only the first, which is why so much of it is useless.

2. Say what a row is

The single most valuable sentence in any spatial metadata record defines the feature: "one polygon per Lower Layer Super Output Area as at the 2021 census, England and Wales only". It resolves the ambiguities that no schema can — whether the layer is complete, what the unit of observation is, and what a missing feature means.

3. Record two dates, not one

The date the data describes and the date the file was made are different, and mixing them is the commonest metadata error. A 2024 survey published in 2026 is 2024 data. Add a third — a review or expiry date — if the dataset goes stale.

4. Write the lineage as a chain

Lineage is not "derived from LiDAR". It is the ordered list of inputs and operations, each with a version: source dataset and its version, the processing steps, the software and its version, the parameters that changed the answer. Provenance and lineage explained for spatial workflows covers how much detail is enough.

5. State accuracy as a number with a method

"High accuracy" means nothing. "RMSE 1.8 m horizontal, assessed against 43 GNSS control points, 2025-06" means something a user can propagate through their own analysis.

6. Put the licence in the record, not in an email

A dataset with no licence is a dataset nobody outside your organisation can safely use. Use an SPDX identifier where one exists so it can be checked mechanically. Open data licences explained for spatial data covers the ones you will meet.

7. Decide where the record lives

Beside the file, inside the file, or in a catalogue — ideally all three, generated from one source. A sidecar is easy to lose; an embedded record travels with the data but not every format has a slot; a catalogue is findable but goes stale. How to read the metadata already inside your spatial files shows what each format can hold.

8. Generate it, do not type it

Extent, CRS, feature count, schema, geometry types and checksums are all computable. Anything a program can derive should be derived at write time, so that only the human fields — abstract, lineage narrative, licence, contact — need writing.

Three panels showing metadata beside the file, inside the file and in a catalogue, with the failure mode of each.
Each location fails differently, which is why the record should be generated into all three.

Code examples

Example 1 — derive everything derivable 

import geopandas as gpd, hashlib, pathlib, datetime, json

def derived_metadata(path, layer=None):
    gdf = gpd.read_file(path, layer=layer)
    b = gdf.total_bounds
    return {
        "file": pathlib.Path(path).name,
        "bytes": pathlib.Path(path).stat().st_size,
        "sha256": hashlib.sha256(pathlib.Path(path).read_bytes()).hexdigest(),
        "features": int(len(gdf)),
        "geometry_types": sorted(gdf.geom_type.dropna().unique().tolist()),
        "crs": gdf.crs.to_string() if gdf.crs else None,
        "bbox": [round(float(v), 6) for v in b],
        "schema": {c: str(t) for c, t in gdf.dtypes.items() if c != gdf.geometry.name},
        "null_counts": {c: int(gdf[c].isna().sum()) for c in gdf.columns if c != gdf.geometry.name},
        "generated": datetime.datetime.now(datetime.timezone.utc).isoformat(timespec="seconds"),
    }

print(json.dumps(derived_metadata("boundaries.gpkg", layer="lsoa"), indent=2)[:400])

Note the checksum caveat: a byte hash of a GeoPackage changes every time you write it, even from identical data. How to checksum spatial datasets so you can prove they match explains why and what to hash instead.

Example 2 — the human fields, in a file next to the data 

import yaml, pathlib

record = {
    "title": "Coastal flood risk zones, East Sussex",
    "feature_definition": "One polygon per contiguous modelled inundation area at the "
                          "1-in-200-year return period. Areas under 100 m² removed.",
    "content_date": "2025-11-01",
    "published_date": "2026-01-20",
    "review_by": "2027-11-01",
    "lineage": [
        "Environment Agency LiDAR Composite DTM 2024, 2 m, tiles TQ40–TQ60",
        "pysheds 0.4 breach_depressions → flowdir(D8) → accumulation",
        "Return period surface from EA fluvial model v7, 200-year",
        "Polygonised with rasterio.features.shapes, simplified 1 m",
    ],
    "accuracy": {"positional_rmse_m": 2.0, "vertical_rmse_m": 0.15,
                 "method": "43 GNSS control points, June 2025"},
    "licence": "OGL-UK-3.0",
    "attribution": "Contains Environment Agency data © Crown copyright and database right 2025",
    "contact": "[email protected]",
}
pathlib.Path("flood_zones.meta.yaml").write_text(yaml.safe_dump(record, sort_keys=False))

Example 3 — refuse to publish without the human fields 

REQUIRED = ["title", "feature_definition", "content_date", "lineage", "licence", "contact"]

def check_metadata(record):
    missing = [k for k in REQUIRED if not record.get(k)]
    if missing:
        raise ValueError(f"metadata incomplete: {missing}")
    if record["licence"] == "unknown":
        raise ValueError("licence must be resolved before publication")
    return True

A gate is the only mechanism that works. Metadata that is optional is metadata that does not exist six months later.

Explanation

Why "documentation" is not the same thing

A README is read by a person who already found the file. Metadata is read by a catalogue, a validator, a pipeline and a stranger, and each of those needs fields rather than sentences. The practical test: if a program cannot decide from your record whether the dataset covers a given date and bounding box, it is documentation.

Why lineage is the field people regret omitting

Every other field can be reconstructed from the data. Lineage cannot — once the person who ran the process has left, the chain of inputs and parameters is unrecoverable, and any question about why a number looks odd becomes unanswerable. It is also the field that makes a result reproducible, which is why scientific metadata standards put it at the centre.

Why the format decides what you can embed

GeoPackage, GeoTIFF and FlatGeobuf all have somewhere to put a dataset-level record; GeoJSON and shapefile do not. A GeoPackage written through GDAL stores dataset metadata in its own gpkg_metadata table; a GeoTIFF stores it as TIFF tags; a shapefile has a .shp.xml sidecar that half the tools ignore. Knowing which of your formats can hold the record decides whether a sidecar is a convenience or the only option.

Why standards matter less than completeness

ISO 19115, STAC, Frictionless and DCAT all express the same nine fields with different names and different amounts of ceremony. Pick the one your catalogue reads and map to the others; a complete record in a plain YAML file beats an empty ISO 19139 XML document, and the second is far more common than the first.

Two panels separating discovery metadata — title, abstract, extent, licence — from use metadata — CRS, schema, units, accuracy and lineage.
A record that can be found and not used is half a record.

Edge cases or notes

  • Units belong in the record. A column called depth with no unit is not usable.
  • Nodata is metadata. A raster whose nodata value is undocumented will be averaged with −9999 in it.
  • Say what is excluded. "England and Wales only" prevents a silent gap in a UK-wide map.
  • Version the record with the data. A metadata edit is a new version of the dataset.
  • Contact a role, not a person. People leave; gis@ does not.
  • Keep the record in the repository. Metadata that lives only in a portal is lost when the portal is replaced.
  • Coordinate precision is not accuracy. Fifteen decimal places on a 2 m survey is noise.
  • An empty field is better than a wrong one. "Unknown" is information; a guessed date is not.

FAQ

What metadata does a spatial dataset need?

Title, a definition of what one feature represents, extent, the date the content describes, lineage, accuracy with its method, licence, attribution and a contact. Those nine answer almost every question a user has.

Is a README enough?

No. A README is prose for someone who already found the file; metadata is structured fields that a catalogue, a validator and a pipeline can act on.

Where should the metadata live?

Ideally in all three places — beside the file, inside it where the format allows, and in a catalogue — generated from one source so they cannot drift apart.

Which fields should I generate automatically?

Extent, CRS, feature count, geometry types, schema, null counts and checksums. Only the narrative fields need writing by hand.

Do I have to use ISO 19115?

Only if your catalogue requires it. A complete record in YAML is far more useful than an empty ISO 19139 document, and the nine fields map cleanly onto every standard.

What is the difference between the content date and the publication date?

The content date is when the data was true on the ground; the publication date is when the file was made. Confusing them makes a 2024 survey look like 2026 data.