Spatial metadata explained: what a dataset must tell you

On this page

Problem statement

Somebody hands you boundaries_final_v3.gpkg. It opens, it draws, the geometry is valid. You still cannot use it, because none of the questions that decide whether it is fit for the job have answers: what do these polygons represent, when were they true, where did they come from, how accurate are they, and are you allowed to publish a map made from them.

Metadata is the record that answers those questions, and it is not documentation. Documentation is prose somebody may read; metadata is structured fields a program can check, a catalogue can index and a pipeline can refuse to run without. The distinction matters because the failure mode is always the same — the person who knew is gone, and the file is still in production.

This guide sets out the fields that earn their place, where each one can live, and which of them your file formats will actually carry — building a real record for Rutland's 15 ward boundaries from the ONS, and testing which formats keep it.

Quick answer

Nine fields answer almost every question anyone asks of a spatial dataset. For the ONS ward boundaries of Rutland:

b = wards.to_crs(4326).total_bounds
metadata = {
    "title":        "Ward boundaries, Rutland, December 2024 (full resolution, clipped)",
    "abstract":     "One polygon per electoral ward in force in December 2024, clipped to the coastline.",
    "extent":       {"bbox": [round(float(v), 4) for v in b], "crs": "EPSG:4326"},
    "temporal":     {"content_date": "2024-12", "published": service_edit_date},
    "lineage":      f"ONS Open Geography Portal, {SERVICE} → query LAD24CD='E06000017' → GeoJSON, EPSG:27700",
    "accuracy":     "BFC: full resolution, clipped to the coastline — the ONS's most detailed boundary set",
    "licence":      "OGL-UK-3.0",
    "attribution":  "Source: Office for National Statistics licensed under the Open Government Licence v.3.0. "
                    "Contains OS data © Crown copyright and database right 2024",
    "contact":      "Office for National Statistics, Open Geography Portal",
}
print(json.dumps({k: metadata[k] for k in ("extent", "temporal")}, indent=None))

{"extent": {"bbox": [-0.8218, 52.5247, -0.4284, 52.7598], "crs": "EPSG:4326"}, "temporal": {"content_date": "2024-12", "published": "2024-12-13"}}

Everything else is elaboration. If a record has these nine and they are true, a stranger can decide whether to use the dataset; if it has forty fields and no lineage, they cannot.

Nine fields, four questions: what is it, when was it true, where did it come from, what may I do with it.

Discovery against use, what a row is, two dates, lineage

1. Separate discovery from use

Metadata has two audiences. A discovery record — title, abstract, extent, keywords, licence — exists so somebody can find the dataset and decide whether to open it. A use record — CRS, schema, units, accuracy, nodata, lineage — exists so somebody who has opened it does not misinterpret it. Catalogues want the first; your pipeline needs the second. Most published metadata has only the first, which is why so much of it is useless.

2. Say what a row is

The single most valuable sentence in any spatial metadata record defines the feature: "one polygon per Lower Layer Super Output Area as at the 2021 census, England and Wales only". It resolves the ambiguities that no schema can — whether the layer is complete, what the unit of observation is, and what a missing feature means.

3. Record two dates, not one

The date the data describes and the date the file was made are different, and mixing them is the commonest metadata error. A 2024 survey published in 2026 is 2024 data. Add a third — a review or expiry date — if the dataset goes stale.

4. Write the lineage as a chain

Lineage is not "derived from LiDAR". It is the ordered list of inputs and operations, each with a version: source dataset and its version, the processing steps, the software and its version, the parameters that changed the answer. Provenance and lineage explained for spatial workflows covers how much detail is enough.

5. State accuracy as a number with a method

"High accuracy" means nothing. "RMSE 1.8 m horizontal, assessed against 43 GNSS control points, 2025-06" means something a user can propagate through their own analysis.

6. Put the licence in the record, not in an email

A dataset with no licence is a dataset nobody outside your organisation can safely use. Use an SPDX identifier where one exists so it can be checked mechanically. Open data licences explained for spatial data covers the ones you will meet.

7. Decide where the record lives

Beside the file, inside the file, or in a catalogue — ideally all three, generated from one source. A sidecar is easy to lose; an embedded record travels with the data but not every format has a slot; a catalogue is findable but goes stale. How to read the metadata already inside your spatial files shows what each format can hold.

8. Generate it, do not type it

Extent, CRS, feature count, schema, geometry types and checksums are all computable. Anything a program can derive should be derived at write time, so that only the human fields — abstract, lineage narrative, licence, contact — need writing.

Each location fails differently, which is why the record should be generated into all three.

Deriving what can be derived for Rutland's wards

The data used on this page

Rutland's wards from the ONS December 2024 full-resolution boundaries (OGL), and the feature service's own description of itself from the ArcGIS REST endpoint.

import datetime
import hashlib
import json
import pathlib
from pathlib import Path

import geopandas as gpd
import pyogrio
import requests
import yaml

SERVICE = ("https://services1.arcgis.com/ESMARspQHYMw9BZ9/arcgis/rest/services/"
           "Wards_December_2024_Boundaries_UK_BFC/FeatureServer/0")
Path("data").mkdir(exist_ok=True)
if not Path("data/wards.geojson").exists():
    Path("data/wards.geojson").write_bytes(requests.get(SERVICE + "/query", params={
        "where": "LAD24CD='E06000017'", "outFields": "WD24NM", "outSR": 27700, "f": "geojson"}, timeout=120).content)
if not Path("data/service.json").exists():
    Path("data/service.json").write_text(requests.get(SERVICE, params={"f": "json"}, timeout=60).text)
wards = gpd.read_file("data/wards.geojson")
service = json.loads(Path("data/service.json").read_text())
service_edit_date = datetime.datetime.fromtimestamp(service["editingInfo"]["dataLastEditDate"] / 1000,
                                                    datetime.timezone.utc).date().isoformat()
print(len(wards), "wards | service data last edited", service_edit_date)

15 wards | service data last edited 2024-12-13

Example 1 — derive everything derivable

def derived_metadata(path, layer=None):
    gdf = gpd.read_file(path, layer=layer)
    b = gdf.total_bounds
    return {
        "file": pathlib.Path(path).name,
        "bytes": pathlib.Path(path).stat().st_size,
        "sha256": hashlib.sha256(pathlib.Path(path).read_bytes()).hexdigest(),
        "features": int(len(gdf)),
        "geometry_types": sorted(gdf.geom_type.dropna().unique().tolist()),
        "crs": gdf.crs.to_string() if gdf.crs else None,
        "bbox": [round(float(v), 6) for v in b],
        "schema": {c: str(t) for c, t in gdf.dtypes.items() if c != gdf.geometry.name},
        "null_counts": {c: int(gdf[c].isna().sum()) for c in gdf.columns if c != gdf.geometry.name},
        "generated": datetime.datetime.now(datetime.timezone.utc).isoformat(timespec="seconds"),
    }

wards.to_file("data/wards.gpkg", layer="wards")
derived = derived_metadata("data/wards.gpkg", layer="wards")
print(json.dumps({k: v for k, v in derived.items() if k not in ("sha256", "generated")}, indent=1))

{
 "file": "wards.gpkg",
 "bytes": 528384,
 "features": 15,
 "geometry_types": [
  "Polygon"
 ],
 "crs": "EPSG:27700",
 "bbox": [
  479688.4991,
  292648.41,
  506295.5998,
  318927.6973
 ],
 "schema": {
  "WD24NM": "str"
 },
 "null_counts": {
  "WD24NM": 0
 }
}

Note the checksum caveat: a byte hash of a GeoPackage changes every time you write it, even from identical data. How to checksum spatial datasets so you can prove they match explains why and what to hash instead.

Example 2 — the human fields, in a file next to the data

record = {
    "title": metadata["title"],
    "feature_definition": "One polygon per electoral ward in Rutland in force in December 2024 — "
                          "15 wards, clipped to the coastline, from the ONS full-resolution set.",
    "content_date": "2024-12",
    "published_date": service_edit_date,
    "review_by": "2025-12",                           # the ONS publishes a new ward set each year
    "lineage": [
        f"ONS Open Geography Portal feature service {SERVICE}",
        "query where LAD24CD = 'E06000017', outFields WD24NM, outSR 27700, f=geojson",
        f"written to GeoPackage with GeoPandas {gpd.__version__}",
    ],
    "accuracy": {"resolution": "full (BFC)", "method": "as published; not independently assessed"},
    "licence": "OGL-UK-3.0",
    "attribution": metadata["attribution"],
    "contact": metadata["contact"],
}
Path("data/wards.meta.yaml").write_text(yaml.safe_dump(record, sort_keys=False, allow_unicode=True))
print(Path("data/wards.meta.yaml").read_text()[:300])

title: Ward boundaries, Rutland, December 2024 (full resolution, clipped)
feature_definition: One polygon per electoral ward in Rutland in force in December
  2024 — 15 wards, clipped to the coastline, from the ONS full-resolution set.
content_date: 2024-12
published_date: '2024-12-13'
review_by: 20

Note what the accuracy field says: "not independently assessed". That is true, and it is more useful than a number nobody measured. The service itself offers little help — its own description and copyright fields are empty:

print({key: service[key] for key in ("name", "description", "copyrightText")})

{'name': 'WD_DEC_24_UK_BFC', 'description': '', 'copyrightText': ''}

The record has to be written by whoever takes the data, because the source did not write it.

Example 3 — refuse to publish without the human fields

REQUIRED = ["title", "feature_definition", "content_date", "lineage", "licence", "contact"]

def check_metadata(record):
    missing = [k for k in REQUIRED if not record.get(k)]
    if missing:
        raise ValueError(f"metadata incomplete: {missing}")
    if record["licence"] == "unknown":
        raise ValueError("licence must be resolved before publication")
    return True

print(check_metadata(record))
try:
    check_metadata({**record, "lineage": [], "licence": "unknown"})
except ValueError as exc:
    print(exc)

True
metadata incomplete: ['lineage']

A gate is the only mechanism that works. Metadata that is optional is metadata that does not exist six months later.

Example 4 — which formats keep an embedded record

embedded = {"TITLE": record["title"], "DESCRIPTION": record["feature_definition"], "LICENSE": record["licence"]}
for driver, suffix in (("GPKG", "gpkg"), ("FlatGeobuf", "fgb"), ("GeoJSON", "geojson"), ("ESRI Shapefile", "shp")):
    pyogrio.write_dataframe(wards, f"data/embedded.{suffix}", driver=driver, layer_metadata=embedded)
    kept = pyogrio.read_info(f"data/embedded.{suffix}").get("layer_metadata") or {}
    print(f"{driver:15} kept: {sorted(k for k in kept if k in embedded) or 'nothing'}")

GPKG            kept: ['DESCRIPTION', 'LICENSE', 'TITLE']
FlatGeobuf      kept: ['DESCRIPTION', 'LICENSE', 'TITLE']
GeoJSON         kept: nothing
ESRI Shapefile  kept: nothing

GeoPackage (in its gpkg_metadata tables) and FlatGeobuf (in its header) kept the record. GeoJSON and shapefile accepted the write and silently dropped it — for those, a sidecar is not a convenience but the only option.

Why documentation is not metadata

Why "documentation" is not the same thing

A README is read by a person who already found the file. Metadata is read by a catalogue, a validator, a pipeline and a stranger, and each of those needs fields rather than sentences. The practical test: if a program cannot decide from your record whether the dataset covers a given date and bounding box, it is documentation.

Why lineage is the field people regret omitting

Every other field can be reconstructed from the data. Lineage cannot — once the person who ran the process has left, the chain of inputs and parameters is unrecoverable, and any question about why a number looks odd becomes unanswerable. It is also the field that makes a result reproducible, which is why scientific metadata standards put it at the centre.

Why the format decides what you can embed

GeoPackage, GeoTIFF and FlatGeobuf all have somewhere to put a record; GeoJSON and shapefile do not, and example 4 showed GDAL dropping one without a word. A GeoPackage written through GDAL stores metadata in its own gpkg_metadata tables; a GeoTIFF stores it as TIFF tags; a shapefile has at best an ArcGIS .shp.xml sidecar that most other tools ignore. Knowing which of your formats can hold the record decides whether a sidecar is a convenience or the only option.

Why standards matter less than completeness

ISO 19115, STAC, Frictionless and DCAT all express the same nine fields with different names and different amounts of ceremony. Pick the one your catalogue reads and map to the others; a complete record in a plain YAML file beats an empty ISO 19139 XML document, and the second is far more common than the first.

A record that can be found and not used is half a record.

Units, nodata and exclusions

Units belong in the record. A column called depth with no unit is not usable.
Nodata is metadata. A raster whose nodata value is undocumented will be averaged with −9999 in it.
Say what is excluded. "England and Wales only" prevents a silent gap in a UK-wide map.
Version the record with the data. A metadata edit is a new version of the dataset.
Contact a role, not a person. People leave; gis@ does not.
Keep the record in the repository. Metadata that lives only in a portal is lost when the portal is replaced.
Coordinate precision is not accuracy. Fifteen decimal places on a 2 m survey is noise.
An empty field is better than a wrong one. "Unknown" is information; a guessed date is not.

Internal links

How to write a metadata record for a dataset in Python — generating the record
Metadata standards compared: ISO 19115, STAC and Frictionless — choosing a shape for it
Provenance and lineage explained for spatial workflows — the field nobody can reconstruct
How to read the metadata already inside your spatial files — what you already have
Open data licences explained for spatial data — the field that decides publication
Dataset versioning explained — which version produced the map
Spatial data quality dimensions — what "accuracy" decomposes into
Metadata disappears when you convert the file — the commonest way records are lost

FAQ

What metadata does a spatial dataset need?

Title, a definition of what one feature represents, extent, the date the content describes, lineage, accuracy with its method, licence, attribution and a contact. Those nine answer almost every question a user has.

Is a README enough?

No. A README is prose for someone who already found the file; metadata is structured fields that a catalogue, a validator and a pipeline can act on.

Where should the metadata live?

Ideally in all three places — beside the file, inside it where the format allows, and in a catalogue — generated from one source so they cannot drift apart.

Which fields should I generate automatically?

Extent, CRS, feature count, geometry types, schema, null counts and checksums. Only the narrative fields need writing by hand.

Do I have to use ISO 19115?

Only if your catalogue requires it. A complete record in YAML is far more useful than an empty ISO 19139 document, and the nine fields map cleanly onto every standard.

What is the difference between the content date and the publication date?

The content date is when the data was true on the ground; the publication date is when the file was made. Confusing them makes a 2024 survey look like 2026 data.