🧭 Concept OpenStreetMap data Files & formats Notebook By · Updated

OpenStreetMap File Formats Explained: PBF, XML and the Rest

On this page

Problem statement

OpenStreetMap data comes in several file formats. Geofabrik and planet.openstreetmap.org offer .osm.pbf, older tools write .osm XML, and others produce .osm.bz2, .o5m, OPL, or GeoJSON and GeoParquet exports. They differ by an order of magnitude in size and in what they contain. The OSM formats store raw nodes, ways and relations. GIS formats store finished geometries with a few chosen tags. Picking the wrong one means downloading ten times more data than needed, or losing the topology a routing engine needs.

This page converts the Washington, D.C. extract of 1 January 2026 between OSM formats with pyosmium and compares their sizes and contents. It then exports the same features to GeoJSON and GeoParquet, to show how the raw OSM model and a GIS layer differ.

Quick answer

pyosmium writes any OSM format from any other; the file extension chooses the format:

for ext in ("osm", "osm.bz2", "opl"):
    out = Path(f"data/dc.{ext}"); out.unlink(missing_ok=True)
    with osmium.SimpleWriter(str(out)) as w:
        for obj in osmium.FileProcessor(str(PBF)):
            w.add(obj)
sizes = {f.name: f.stat().st_size / 1e6 for f in [PBF] + [Path(f"data/dc.{e}") for e in ("osm", "osm.bz2", "opl")]}
for name, mb in sizes.items():
    print(f"{name:>36}: {mb:7.1f} MB ({mb / sizes[PBF.name]:.1f}x the PBF)")
 district-of-columbia-260101.osm.pbf:    20.3 MB (1.0x the PBF)
                              dc.osm:   384.7 MB (18.9x the PBF)
                          dc.osm.bz2:    29.4 MB (1.4x the PBF)
                              dc.opl:   231.4 MB (11.4x the PBF)

The same data is 384.7 MB as XML, 18.9 times the PBF, and 231.4 MB as OPL. Compressing the XML with bzip2 brings it to 29.4 MB, still 1.4 times the PBF, and it must be decompressed and parsed as text to be read.

The PBF is 20.3 MB.

Size of the D.C. OpenStreetMap extract in four OSM formats. the same data in four formats.osm.pbf20.3 MB.osm.bz229.4 MB.opl231.4 MB.osm (XML)384.7 MB0100 MB200 MB300 MB400 MB1,917,699 nodes, 271,679 ways, 5,211 relations in each spatialworkflow.io
The same data in four formats.

The same content in every format, XML spelled out, GIS exports differ

1. The same content in every OSM format

The conversion is lossless: count the elements in the PBF and in the XML:

def count(path):
    n = {"n": 0, "w": 0, "r": 0}
    for obj in osmium.FileProcessor(str(path)):
        n[obj.type_str()] += 1
    return n
print(f"PBF: {count(PBF)}")
print(f"XML: {count('data/dc.osm')}")
PBF: {'n': 1917699, 'w': 271679, 'r': 5211}
XML: {'n': 1917699, 'w': 271679, 'r': 5211}

Both files hold exactly the same 1,917,699 nodes, 271,679 ways and 5,211 relations: the conversion lost nothing.

OSM formats and GIS formats hold different things. raw data or finished featuresGIS exportGeoJSON, GeoPackage, GeoParquetgeometries + chosen tagsOSM formatsPBF, XML, OPL, o5mnodes, ways, relations, all tagsKeep the PBF as the source spatialworkflow.io
Raw data or finished features.

2. What XML spells out

XML repeats every tag name and attribute in text. Look at the start of the file:

with open("data/dc.osm") as f:
    head = [next(f).rstrip() for _ in range(6)]
print("\n".join(line[:110] for line in head))
<?xml version='1.0' encoding='UTF-8'?>
<osm version="0.6" generator="libosmium/2.23.1">
  <node id="281072" version="3" timestamp="2025-05-05T23:45:21Z" lat="38.9358304" lon="-77.108256">
    <tag k="crossing" v="traffic_signals"/>
    <tag k="crossing:continuous" v="yes"/>
    <tag k="highway" v="crossing"/>

Every node in the XML carries its ID, version, timestamp and coordinates as text attributes, and every tag repeats its key and value in full. That is why the file is so large.

3. A GIS export holds something different

Export the amenities as finished geometries with two tags, to GeoJSON and GeoParquet:

fp = (osmium.FileProcessor(str(PBF)).with_areas().with_filter(osmium.filter.EntityFilter(osmium.osm.NODE | osmium.osm.AREA))
      .with_filter(osmium.filter.KeyFilter("amenity")).with_filter(osmium.filter.GeoInterfaceFilter(tags=["amenity", "name"])))
A = gpd.GeoDataFrame.from_features(fp, crs=4326)
A.to_file("data/amenity.geojson", driver="GeoJSON"); A.to_parquet("data/amenity.parquet")
print(f"{len(A):,} amenity features: GeoJSON {Path('data/amenity.geojson').stat().st_size / 1e6:.1f} MB, GeoParquet {Path('data/amenity.parquet').stat().st_size / 1e6:.1f} MB")
16,802 amenity features: GeoJSON 6.5 MB, GeoParquet 2.1 MB

As a GIS export, the 16,802 amenity features with two tags each take 6.5 MB as GeoJSON and 2.1 MB as GeoParquet. That is small, but only because most of the raw data was left behind.

How PBF stays small

The data used on this page

The Geofabrik OpenStreetMap extract for the District of Columbia dated 1 January 2026 (© OpenStreetMap contributors, ODbL), converted with pyosmium 4.

from pathlib import Path

import geopandas as gpd
import osmium
import requests

Path("data").mkdir(exist_ok=True)
PBF = Path("data/district-of-columbia-260101.osm.pbf")
if not PBF.exists():
    r = requests.get("https://download.geofabrik.de/north-america/us/district-of-columbia-260101.osm.pbf", timeout=600)
    r.raise_for_status()
    PBF.write_bytes(r.content)
print(f"{PBF.name}: {PBF.stat().st_size / 1e6:.1f} MB")
district-of-columbia-260101.osm.pbf: 20.3 MB

Example 1: how PBF stays small

PBF stores coordinates as integers relative to the previous node and tag strings once per block. Measure the bytes per node implied by the file:

n = count(PBF)
print(f"bytes in the PBF per element: {PBF.stat().st_size / sum(n.values()):.1f}; per node if nodes took all of it: {PBF.stat().st_size / n['n']:.1f}")
print(f"XML bytes per element: {Path('data/dc.osm').stat().st_size / sum(n.values()):.0f}")
bytes in the PBF per element: 9.3; per node if nodes took all of it: 10.6
XML bytes per element: 175

The PBF averages 9.3 bytes per element, under 11 bytes per node even if nodes took up the whole file, against 175 bytes per element in XML.

Bytes per element and per feature in different formats. bytes per elementformatholdssizePBF2,194,589 elements9.3 B eachXML2,194,589 elements175 B eachGeoJSON16,802 amenity features6.5 MBGeoParquet16,802 amenity features2.1 MBthe GIS exports hold two tags per featureD.C. extract, 1 January 2026 spatialworkflow.io
Bytes per element.

All formats store the raw data model

How a PBF file encodes OpenStreetMap data compactly. inside a PBF blockblock ofelementsthousands at a timestring tableeach tag stringoncedelta codingcoordinates assmall integersProtocolBuffersvarint encodingzlib per blockread in parallel9.4 bytes per element in the D.C. extract spatialworkflow.io
Inside a PBF block.

The OSM formats all store the same thing, the raw data model: nodes with coordinates, ways as lists of node IDs, relations as lists of members, every element with its tags, version and timestamp. XML (.osm) was the original format and is still what the editing API speaks. It is readable and verbose, and usually compressed with bzip2 for distribution. PBF (.osm.pbf) encodes the same content with Protocol Buffers in independently compressed blocks: coordinates as delta-encoded integers, tag strings in a per-block table. That makes it smaller even than bzip2-compressed XML, and it is designed to be read quickly, in parallel blocks, which is why Geofabrik and the planet dumps use it. OPL is a line-per-object text format convenient for grep and debugging. o5m, another compact binary format, is used by a few tools.

GeoJSON, GeoPackage and GeoParquet are not OSM formats. They hold finished features: geometries assembled from the raw data, with the tags someone chose to keep. That makes them convenient for analysis, but they lose what the OSM formats keep, including shared nodes between ways, relation membership, versions, and the tags that were dropped. Routing graphs, history and re-tagging need the OSM formats. Maps and statistics usually want a GIS export. A good workflow keeps the PBF as the source and generates GIS layers from it.

Planet files, history and change files

Choosing an OpenStreetMap file format. which format?download.osm.pbfdebug, grep.oplediting API, old tools.osm XMLanalysis layersGeoParquet, GeoPackageupdates.osc.gz change filespyosmium converts between OSM formats spatialworkflow.io
Which format?
  • Planet scale: the full planet is tens of gigabytes as PBF and larger again as compressed XML, which is also far slower to parse; never download XML for large areas.
  • History files: .osh.pbf files hold every version of every element; readers must be told to handle history.
  • Change files: .osc.gz diffs update an extract from one state to the next without downloading it again.
  • Locations: converting PBF to GIS formats needs node locations for ways; pyosmium's with_locations() handles it.

FAQ

What is an .osm.pbf file?

OpenStreetMap data in the Protocol Buffer Binary Format: the full raw data (nodes, ways, relations, tags) compressed into blocks, several times smaller than XML.

What is the difference between .osm and .osm.pbf?

Both hold the same data; .osm is XML text, .osm.pbf is a compact binary encoding that is smaller and faster to read.

How do I convert PBF to XML or GeoJSON?

pyosmium or osmium-tool convert between OSM formats; for GeoJSON, read features with pyosmium or GDAL and write them with GeoPandas.

Which OSM format should I download?

PBF, almost always; convert to GIS formats locally for the features you need.