How to read the metadata already inside your spatial files #

Problem statement #

Before writing a metadata record, find out what the file already tells you. Most spatial formats carry more than people realise — a GeoPackage has a whole metadata table, a GeoTIFF has tags, a Parquet file has a footer describing its own geometry column — and reading it is faster than asking the person who made it.

It also tells you what you are about to lose. A conversion that drops the record is invisible at the time and expensive later, and the only way to know is to look at both sides.

This guide reads every slot, format by format, with what each one actually holds.

Quick answer #

import geopandas as gpd, rasterio, pyarrow.parquet as pq, json, sqlite3
from osgeo import gdal
gdal.UseExceptions()

# Vector, any GDAL format: dataset- and layer-level metadata
ds = gdal.OpenEx("data.gpkg")
print("dataset:", ds.GetMetadata())
print("layers:", [ds.GetLayer(i).GetName() for i in range(ds.GetLayerCount())])

# Raster
with rasterio.open("scene.tif") as src:
    print(src.profile, src.tags(), src.descriptions, src.units)

# GeoParquet
md = pq.read_metadata("data.parquet").metadata
print([k.decode() for k in md])
print(json.loads(md[b"geo"]))

On a GeoParquet file written by GeoPandas, the footer keys are ARROW:schema, geo and pandas, and the geo block holds primary_column, columns, version and creator — with encoding, crs, geometry_types and bbox per geometry column.

Step-by-step solution #

1. GeoPackage — a real metadata table #

GDAL stores dataset metadata in the GeoPackage's own gpkg_metadata and gpkg_metadata_reference tables, as defined by the standard. Writing four items produced one row in each table, with the payload stored as XML under a GDAL-specific standard URI:

('dataset', 'http://gdal.org', 'text/xml',
 '<GDALMultiDomainMetadata>\n  <Metadata>\n    <MDI key="ABSTRACT">…')

You can read it through GDAL, or straight out of SQLite if you would rather not depend on the bindings.

2. GeoTIFF — tags, per band and per dataset #

src.tags() returns the dataset tags; src.tags(1) the tags of band 1. Rasterio adds AREA_OR_POINT itself, so a file with four of your own tags reads back five.

3. GeoParquet — the footer #

Arrow stores arbitrary key–value metadata in the Parquet footer. The geo key holds the GeoParquet block; pandas holds the DataFrame's index and dtypes. You can add your own key at write time and read it back without a spatial library.

4. Shapefile — almost nothing #

The .prj holds the CRS as WKT, the .dbf holds field names truncated to ten characters, the .cpg holds the encoding. Anything else lives in a .shp.xml sidecar that many tools do not write and more do not read.

5. GeoJSON — nothing standard #

The format has no dataset-level metadata slot. A GeoJSON written by GeoPandas has the top-level members type, name, crs and features. You can add your own members — GeoPandas still reads the file, and a test with added title and license members read back 40 features without complaint — but GDAL will not report them as dataset metadata, and any tool that rewrites the file will drop them.

6. Compare the two sides of a conversion #

The reason to read all of this is to know what a conversion costs. Run the reader before and after and diff the result.

7. Recover what you can from the data itself #

Even with no record at all, the file tells you the CRS, the extent, the schema, the geometry types and the null pattern. That is most of the derived half of a record, and it is often enough to make a dataset usable again.

Code examples #

Example 1 — one function that reports every slot #

import json, pathlib, sqlite3
import geopandas as gpd, rasterio, pyarrow.parquet as pq
from osgeo import gdal
gdal.UseExceptions()

def read_all_metadata(path):
    path = pathlib.Path(path)
    out = {"file": path.name, "bytes": path.stat().st_size, "suffix": path.suffix}

    if path.suffix in {".tif", ".tiff"}:
        with rasterio.open(path) as src:
            out["dataset_tags"] = src.tags()
            out["band_tags"] = {i: src.tags(i) for i in range(1, src.count + 1)}
            out["descriptions"] = list(src.descriptions)
            out["units"] = list(src.units)
            out["profile"] = {k: str(v) for k, v in src.profile.items()}
        return out

    if path.suffix == ".parquet":
        md = pq.read_metadata(path).metadata or {}
        out["footer_keys"] = [k.decode() for k in md]
        if b"geo" in md:
            out["geo"] = json.loads(md[b"geo"])
        return out

    ds = gdal.OpenEx(str(path))
    out["dataset_metadata"] = ds.GetMetadata()
    out["domains"] = ds.GetMetadataDomainList() or []
    out["layers"] = []
    for i in range(ds.GetLayerCount()):
        layer = ds.GetLayer(i)
        defn = layer.GetLayerDefn()
        out["layers"].append({
            "name": layer.GetName(),
            "features": layer.GetFeatureCount(),
            "geometry": gdal.ogr.GeometryTypeToName(defn.GetGeomType()),
            "metadata": layer.GetMetadata(),
            "fields": [(defn.GetFieldDefn(j).GetName(),
                        defn.GetFieldDefn(j).GetTypeName()) for j in range(defn.GetFieldCount())],
        })
    return out

Example 2 — read GeoPackage metadata without GDAL bindings #

import sqlite3, xml.etree.ElementTree as ET

def gpkg_metadata(path):
    con = sqlite3.connect(path)
    try:
        tables = {r[0] for r in con.execute(
            "select name from sqlite_master where type='table'")}
        if "gpkg_metadata" not in tables:
            return {}
        rows = con.execute(
            "select md_scope, md_standard_uri, mime_type, metadata from gpkg_metadata").fetchall()
    finally:
        con.close()

    out = []
    for scope, uri, mime, payload in rows:
        entry = {"scope": scope, "standard": uri, "mime": mime}
        if mime == "text/xml" and payload.lstrip().startswith("<GDALMultiDomainMetadata"):
            entry["items"] = {e.get("key"): e.text
                              for e in ET.fromstring(payload).iter("MDI")}
        else:
            entry["raw"] = payload[:500]
        out.append(entry)
    return out

Useful in a container without GDAL's Python bindings, and useful for confirming that what GDAL wrote is what the GeoPackage standard expects.

Example 3 — diff the metadata across a conversion #

def conversion_report(src_path, dst_path):
    a, b = read_all_metadata(src_path), read_all_metadata(dst_path)
    lost = {k: v for k, v in (a.get("dataset_metadata") or {}).items()
            if k not in (b.get("dataset_metadata") or {})}
    import geopandas as gpd
    ga, gb = gpd.read_file(src_path), gpd.read_file(dst_path)
    return {
        "dataset_metadata_lost": lost,
        "columns_renamed": {x: y for x, y in zip(ga.columns, gb.columns) if x != y},
        "dtype_changes": {c: (str(ga[c].dtype), str(gb[c].dtype))
                          for c in ga.columns if c in gb.columns
                          and str(ga[c].dtype) != str(gb[c].dtype)},
        "geometry_types": (sorted(ga.geom_type.unique()), sorted(gb.geom_type.unique())),
        "crs_same": (ga.crs == gb.crs),
    }

Run it once on your own pipeline and you will find at least one surprise. Metadata disappears when you convert the file collects the common ones.

Explanation #

Why GDAL is the right reader for the vector side #

The GDAL abstraction gives you the same three levels — dataset, layer, field — for every driver, so one function covers GeoPackage, FileGDB, GML, shapefile and the rest. Reading each format with its own library means writing the same code five times and missing the slots you did not know about.

Why the GeoPackage record is XML inside a table #

The GeoPackage standard defines a metadata table but not a metadata format: the md_standard_uri column says which standard the payload follows. GDAL writes its own GDALMultiDomainMetadata XML with a http://gdal.org URI. A record written by a different tool may be ISO 19139 instead — read the URI before assuming.

Why the Parquet footer is the nicest of the lot #

It is key–value metadata attached to the file by the format itself, readable without any spatial library, and it survives copying and range reads. GeoParquet uses it for the CRS and geometry encoding; there is nothing stopping you adding a dataset key with your own record, which then travels inside the file.

Why GeoJSON is a problem worth planning around #

It is the most common interchange format and the only common one with no metadata slot at all. Extra top-level members survive a read by GeoPandas but are dropped by anything that rewrites the file, and GDAL reports no dataset metadata for them. For GeoJSON deliveries, ship the sidecar and accept that it can be separated.

Edge cases or notes #

• AREA_OR_POINT is always there. Rasterio adds it; it is not one of yours.
• Band descriptions are not tags. src.descriptions and src.tags(i) are different slots.
• Metadata domains exist. GetMetadata("IMAGE_STRUCTURE") holds compression and interleaving.
• FileGDB has rich metadata and it is XML, per item.
• NetCDF attributes are metadata. Global and per-variable; read both.
• A .shp.xml may exist. Check for it before assuming a shapefile has nothing.
• Layer metadata is separate from dataset metadata in GDAL; read both.
• Reading is cheap. Do it in the pipeline, not only when something has gone wrong.

Internal links #

• Metadata disappears when you convert the file — the diff this makes possible
• How to store metadata inside a GeoPackage — writing the slot you just read
• How to embed provenance tags in a GeoTIFF — the raster slot
• How to write a metadata record for a dataset in Python — assembling the full record
• GIS file formats compared — the wider trade-offs between these formats
• GDAL and OGR explained — the abstraction the reader uses
• GeoParquet and columnar storage explained — what is in the footer
• How to explore an unfamiliar spatial dataset in Python — the next step when there is no record

FAQ #

How do I read metadata from a GeoPackage in Python? #

Open it with GDAL and call GetMetadata(), or read the gpkg_metadata table directly with sqlite3. GDAL stores its items as GDALMultiDomainMetadata XML in that table.

Where does a GeoTIFF keep its metadata? #

In TIFF tags, at dataset and band level. src.tags() and src.tags(band) in rasterio; band descriptions and units are separate properties.

Does GeoJSON have a metadata slot? #

No. You can add top-level members and GeoPandas will still read the file, but GDAL reports no dataset metadata and any tool that rewrites the file will drop them.

What is in a GeoParquet footer? #

Key–value metadata including a geo block with the primary geometry column, its encoding, CRS, geometry types and bbox, plus Arrow and pandas schema blocks.

Can I read all of this without GDAL's Python bindings? #

Mostly. GeoPackage metadata is readable with sqlite3, Parquet with pyarrow, GeoTIFF with rasterio. GDAL bindings help for the less common drivers.

Why read metadata before converting a file? #

So you can tell what the conversion cost. Running the same reader on both sides is the only reliable way to see what a driver dropped.