Fiona vs pyogrio: How GeoPandas Reads and Writes Files
On this page
Problem statement
The same call, two engines, very different behaviour — measured on the Land Registry's 338,611 Birmingham parcels, a 194 MB GeoPackage:
engine="fiona" 15.02 s
engine="pyogrio" 1.72 s
(best of three, in the benchmark below).
And arguments that one engine honours and the other silently ignores:
gpd.read_file(path, engine="pyogrio", skip_features=10, max_features=5) # 5 rows
gpd.read_file(path, engine="fiona", skip_features=10, max_features=5) # every row, no warningGeoPandas has two I/O backends. Both call GDAL; they differ in how, and that difference shows up as a factor of eight or nine in speed, an argument list that only partly overlaps, and different types for the same columns. Since GeoPandas 1.0 the default is pyogrio, which means code written against Fiona's behaviour can change under you on upgrade.
Quick answer
Both wrap GDAL; the difference is per-feature Python versus vectorised bulk transfer:
- pyogrio reads whole columns into NumPy arrays in C — fast, and the default since GeoPandas 1.0
- Fiona yields one Python dict per feature — slower, but streamable and flexible
- both accept
columns=,where=,bbox=androws=; only pyogrio acts onskip_features=,max_features=andsql=— Fiona ignores them without a word - Fiona is the better fit when you need to stream millions of features without materialising a frame
- You can force either engine per call, or globally
import geopandas as gpd
# per call
gdf = gpd.read_file("data/raw/parcels.gpkg", engine="pyogrio")
gdf = gpd.read_file("data/raw/parcels.gpkg", engine="fiona")
# globally, e.g. while migrating an old codebase
gpd.options.io_engine = "pyogrio"
# what you actually have
import pyogrio, fiona
print("pyogrio", pyogrio.__version__, "GDAL", pyogrio.__gdal_version_string__)
print("fiona ", fiona.__version__, "GDAL", fiona.__gdal_version__)pyogrio 0.13.0 GDAL 3.12.4
fiona 1.10.1 GDAL 3.12.2
Each engine's pip wheel carries its own GDAL, and they need not be the same version.
The practical rule: use pyogrio for everything unless you are streaming feature-by-feature, in which case Fiona's iterator model is the right tool.
Two paths to the same GDAL
Speed, arguments and behaviour, engine by engine
Why pyogrio is faster
Fiona's model is a Python iterator of GeoJSON-like dicts. Reading a million features means constructing a million dicts, each with a nested coordinate list, then building a GeoDataFrame from them. Every one of those objects is allocated, populated and garbage-collected by CPython.
import fiona
with fiona.open("data/raw/parcels.gpkg") as src:
feature = next(iter(src))
print(type(feature))
print(dict(feature["properties"]))
print(feature["geometry"]["type"])<class 'fiona.model.Feature'>
{'gml_id': 'PREDEFINED.fid--4959b20f_186af99ef3e_6c49', 'INSPIREID': 44350822, 'LABEL': 44350822, 'NATIONALCADASTRALREFERENCE': 44350822, 'VALIDFROM': '2010-04-14T08:24:09.559000+00:00', 'BEGINLIFESPANVERSION': '2010-06-24T12:43:14.733000+00:00'}
Polygon
pyogrio asks GDAL for whole columns and copies them into NumPy arrays in one C-level pass, converting geometry to WKB in bulk and letting Shapely 2 build the geometry array vectorised.
import pyogrio
# the low-level view: arrays, not features
meta, fids, geometry, fields = pyogrio.raw.read("data/raw/parcels.gpkg")
print(type(geometry), geometry.dtype, len(geometry)) # ndarray of WKB
print([f.dtype for f in fields][:4])<class 'numpy.ndarray'> object 338611
[dtype('O'), dtype('int32'), dtype('int32'), dtype('int32')]
raw.read returns four things — metadata, feature ids (None unless asked for), geometry and fields; unpacking it into three raises ValueError. That is the whole story: one bulk copy instead of a third of a million small ones, which is where the factor of nine comes from here.
The arguments, and which engine honours them
import geopandas as gpd
path = "data/raw/parcels.gpkg"
calls = {
"columns=": dict(columns=["INSPIREID"]),
"where=": dict(where="INSPIREID > 50000000"),
"bbox=": dict(bbox=(405000, 285000, 410000, 290000)),
"rows=": dict(rows=slice(100_000, 100_005)),
"skip_features= / max_features=": dict(skip_features=100_000, max_features=5),
"sql=": dict(sql="SELECT INSPIREID, geom FROM parcels LIMIT 5", sql_dialect="SQLITE"),
}
for label, kwargs in calls.items():
counts = []
for engine in ("pyogrio", "fiona"):
result = gpd.read_file(path, engine=engine, **kwargs)
counts.append(f"{len(result):>7,} rows, {len(result.columns)} cols")
print(f"{label:31} pyogrio {counts[0]} fiona {counts[1]}")columns= pyogrio 338,611 rows, 2 cols fiona 338,611 rows, 2 cols
where= pyogrio 45,407 rows, 7 cols fiona 45,407 rows, 7 cols
bbox= pyogrio 27,543 rows, 7 cols fiona 27,543 rows, 7 cols
rows= pyogrio 5 rows, 7 cols fiona 5 rows, 7 cols
skip_features= / max_features= pyogrio 5 rows, 7 cols fiona 338,611 rows, 7 cols
sql= pyogrio 5 rows, 2 cols fiona 338,611 rows, 7 cols
columns=, where= and bbox= behave the same in both, with GeoPandas passing them through to Fiona's own filtering. rows= is the portable way to take a window. skip_features, max_features and sql are pyogrio's own arguments, and Fiona drops them without an error or a warning — a chunked reader written for pyogrio and run with the Fiona engine reads the whole file into every chunk.
What Fiona still does better
Streaming. Fiona hands you features one at a time and never builds a frame, so memory stays flat regardless of file size — and it works on formats where row offsets are expensive.
import fiona
from shapely.geometry import shape
with fiona.open("data/raw/parcels.gpkg") as src:
print(src.crs, src.schema["geometry"], len(src))
total = 0
for feature in src: # constant memory
total += shape(feature["geometry"]).area
print(f"{total / 1e6:,.1f} km² of registered parcels without loading a frame")EPSG:27700 Polygon 338611
210.4 km² of registered parcels without loading a frame
Measured in separate processes, streaming the Birmingham file with Fiona peaked at 190 MB for the whole Python process; reading it into a GeoDataFrame peaked at 755 MB with pyogrio and 876 MB with Fiona (Example 5).
It also exposes the schema as a first-class object, which is handy when you are writing a format converter that must preserve field types exactly:
import os
os.makedirs("out", exist_ok=True)
with fiona.open("data/raw/parcels.gpkg") as src:
schema = src.schema
print(schema["properties"])
with fiona.open("out/copy.gpkg", "w", driver="GPKG",
crs=src.crs, schema=schema) as dst:
dst.writerecords(src)
print(len(gpd.read_file("out/copy.gpkg", columns=[])), "features copied"){'gml_id': 'str', 'INSPIREID': 'int32', 'LABEL': 'int32', 'NATIONALCADASTRALREFERENCE': 'int32', 'VALIDFROM': 'datetime', 'BEGINLIFESPANVERSION': 'datetime'}
338611 features copied
Behaviour differences that bite
import geopandas as gpd
a = gpd.read_file(path, engine="fiona")
b = gpd.read_file(path, engine="pyogrio")
print(a.dtypes.equals(b.dtypes))
print(set(a.columns) ^ set(b.columns))
print(a.crs == b.crs, bool(a.geometry.geom_equals_exact(b.geometry, 0).all()))
for column in ("INSPIREID", "VALIDFROM"):
print(f"{column:10} fiona {str(a[column].dtype):22} pyogrio {b[column].dtype}")
print(repr(a["VALIDFROM"].iloc[0]), "|", repr(b["VALIDFROM"].iloc[0]))False
set()
True True
INSPIREID fiona int64 pyogrio int32
VALIDFROM fiona str pyogrio datetime64[ms, UTC]
'2010-04-14T08:24:09.559000+00:00' | Timestamp('2010-04-14 08:24:09.559000+0000', tz='UTC')
Same columns, same CRS, identical geometry — and different types. The ones worth knowing:
- integers — pyogrio keeps the field's declared width (
int32here); Fiona widens toint64 - datetime — pyogrio parses to
datetime64with the time zone; Fiona returns ISO strings for this GeoPackage, so a date comparison that works with one engine is a string comparison with the other - field order and the FID — pyogrio can return the feature id as a column with
fid_as_index=True - null geometry — both produce
None, but empty vs null handling has differed across versions - encoding — Fiona takes
encoding=; pyogrio also does, but resolves the driver default differently
Any pipeline whose output must be byte-stable should pin both the engine and the versions.
Writing
import geopandas as gpd
gdf = b.head(1000)
gdf.to_file("out/parcels.gpkg", driver="GPKG", engine="pyogrio")
gdf.to_file("out/parcels.gpkg", driver="GPKG", engine="fiona")
# pyogrio exposes driver creation options directly
gdf.to_file("out/parcels.shp", driver="ESRI Shapefile", engine="pyogrio", encoding="utf-8")
gdf.to_file("out/parcels_multi.gpkg", engine="pyogrio", promote_to_multi=True)
print(gdf.geom_type.unique().tolist(), "->", gpd.read_file("out/parcels_multi.gpkg").geom_type.unique().tolist())
# append to an existing layer
gdf.to_file("out/parcels.gpkg", layer="parcels", driver="GPKG", mode="a")
print(len(gpd.read_file("out/parcels.gpkg", layer="parcels")), "rows after appending")['Polygon'] -> ['MultiPolygon']
2000 rows after appending
promote_to_multi=True is a pyogrio convenience that solves the mixed Polygon/MultiPolygon write problem in one argument.
Arrow: the next step up
import geopandas as gpd
gdf = gpd.read_file(path, engine="pyogrio", use_arrow=True)
print(gdf.dtypes.astype(str).to_dict()){'gml_id': 'str', 'INSPIREID': 'int32', 'LABEL': 'int32', 'NATIONALCADASTRALREFERENCE': 'int32', 'VALIDFROM': 'datetime64[ms, UTC]', 'BEGINLIFESPANVERSION': 'datetime64[ms, UTC]', 'geometry': 'geometry'}
With use_arrow=True, pyogrio uses GDAL's Arrow stream interface and avoids another copy. It requires pyarrow and a recent GDAL, and it is the fastest path available today — 0.98 s against 1.72 s on the Birmingham file. For this layer the dtypes came out the same as the default read; columns with missing values are where the Arrow path's nullable types differ.
Measure it on your data
import time
import geopandas as gpd
def bench(path, repeats=3):
for engine in ("fiona", "pyogrio"):
times = []
for _ in range(repeats):
t0 = time.perf_counter()
gdf = gpd.read_file(path, engine=engine)
times.append(time.perf_counter() - t0)
print(f"{engine:8} {min(times):6.2f} s {len(gdf):>9,} features")
t0 = time.perf_counter()
arrow = gpd.read_file(path, engine="pyogrio", use_arrow=True)
print(f"{'arrow':8} {time.perf_counter() - t0:6.2f} s {len(arrow):>9,} features")
t0 = time.perf_counter()
slim = gpd.read_file(path, engine="pyogrio", columns=["INSPIREID"])
print(f"{'columns=':8} {time.perf_counter() - t0:6.2f} s {len(slim):>9,} features")
bench("data/raw/parcels.gpkg")fiona 15.02 s 338,611 features
pyogrio 1.72 s 338,611 features
arrow 0.98 s 338,611 features
columns= 0.83 s 338,611 features
The columns= line beats the full pyogrio read by another factor of two or more — narrowing the read is worth as much as the engine choice.
Reading Birmingham's parcels with both engines
The data used on this page
The Land Registry's INSPIRE index polygons for Birmingham: 338,611 freehold parcels in British National Grid, with integer identifiers and two timestamp fields, saved as a 194 MB GeoPackage. The timings on this page were taken on it with GeoPandas 1.1, pyogrio 0.13 and Fiona 1.10.
import io
import os
import zipfile
import geopandas as gpd
import requests
os.makedirs("data/raw", exist_ok=True)
if not os.path.exists("data/raw/parcels.gpkg"):
reply = requests.Session().get("https://use-land-property-data.service.gov.uk/datasets/"
"inspire/download/Birmingham.zip", timeout=600)
reply.raise_for_status()
zipfile.ZipFile(io.BytesIO(reply.content)).extract("Land_Registry_Cadastral_Parcels.gml")
gpd.read_file("Land_Registry_Cadastral_Parcels.gml").to_file("data/raw/parcels.gpkg", layer="parcels")
os.remove("Land_Registry_Cadastral_Parcels.gml")
info = gpd.read_file("data/raw/parcels.gpkg", rows=slice(0, 1))
print(f"{os.path.getsize('data/raw/parcels.gpkg') / 1e6:.0f} MB;", info.columns.tolist(), info.crs)194 MB; ['gml_id', 'INSPIREID', 'LABEL', 'NATIONALCADASTRALREFERENCE', 'VALIDFROM', 'BEGINLIFESPANVERSION', 'geometry'] EPSG:27700
Example 1: an engine-aware reader
import geopandas as gpd
def read_vector(path, columns=None, where=None, bbox=None, prefer="pyogrio"):
"""Use pyogrio's pushdown features when available, fall back cleanly."""
kwargs = {"bbox": bbox} if bbox else {}
try:
return gpd.read_file(
path, engine=prefer,
**({"columns": columns} if columns else {}),
**({"where": where} if where else {}),
**kwargs,
)
except (ImportError, TypeError, ValueError):
return gpd.read_file(path, engine="fiona",
**({"columns": columns} if columns else {}),
**({"where": where} if where else {}), **kwargs)
print(len(read_vector("data/raw/parcels.gpkg", columns=["INSPIREID"], where="INSPIREID > 50000000")),
"parcels")45407 parcels
An earlier version of this fallback rebuilt the where clause as a pandas query, because Fiona was thought not to support it. It does, through GeoPandas, and the string rewriting turned >= into >==.
Example 2: stream a file too large to load
import fiona
from shapely.geometry import shape
def stream_stats(path, layer=None, key=lambda p: str(p.get("VALIDFROM"))[:4]):
total_area = 0.0
by_class = {}
with fiona.open(path, layer=layer) as src:
crs, n = src.crs, len(src)
for i, feature in enumerate(src, start=1):
geom = shape(feature["geometry"]) if feature["geometry"] else None
if geom is not None:
total_area += geom.area
cls = key(feature["properties"] or {})
by_class[cls] = by_class.get(cls, 0) + 1
if i % 250_000 == 0:
print(f" {i:,}/{n:,}", flush=True)
return {"crs": str(crs), "features": n,
"total_area": round(total_area, 2), "by_class": by_class}
stats = stream_stats("data/raw/parcels.gpkg")
print(stats["features"], "parcels,", f"{stats['total_area'] / 1e6:.1f} km²; first registered by year:",
dict(sorted(stats["by_class"].items())[:4]), "…") 250,000/338,611
338611 parcels, 210.4 km²; first registered by year: {'2001': 2217, '2002': 22350, '2003': 14528, '2004': 1204} …
Memory stays flat no matter how large the file is — the thing pyogrio's bulk model cannot do (Example 5 measures it).
Example 3: chunked reads with pyogrio
import geopandas as gpd
import pyogrio
def chunks(path, size=100_000, columns=None):
total = pyogrio.read_info(path)["features"]
for start in range(0, total, size):
yield gpd.read_file(path, engine="pyogrio", columns=columns,
skip_features=start, max_features=size)
os.makedirs("data/out", exist_ok=True)
for i, part in enumerate(chunks("data/raw/parcels.gpkg", columns=["INSPIREID"])):
part["area_m2"] = part.area # British National Grid: already metres
part.to_parquet(f"data/out/part-{i:05d}.parquet", compression="zstd")
print(i + 1, "chunks,", sum(len(gpd.read_parquet(f"data/out/part-{j:05d}.parquet")) for j in range(i + 1)),
"rows in total")4 chunks, 338611 rows in total
Example 4: pin the engine in a pipeline
import geopandas as gpd
def configure_io(engine: str = "pyogrio") -> None:
"""Fix the I/O engine so behaviour cannot change under a library upgrade."""
gpd.options.io_engine = engine
import pyogrio, fiona
versions = {
"geopandas": gpd.__version__,
"engine": engine,
"pyogrio": pyogrio.__version__,
"fiona": fiona.__version__,
"gdal": pyogrio.__gdal_version_string__,
}
print(" | ".join(f"{k}={v}" for k, v in versions.items()))
return versions
versions = configure_io("pyogrio") # record these in your run metadatageopandas=1.1.4 | engine=pyogrio | pyogrio=0.13.0 | fiona=1.10.1 | gdal=3.12.4
Example 5 — memory, streamed or loaded
import subprocess
import sys
PROBE = """
import sys
import geopandas as gpd, fiona
if sys.argv[1] == "stream":
with fiona.open("data/raw/parcels.gpkg") as src:
n = sum(1 for _ in src)
else:
n = len(gpd.read_file("data/raw/parcels.gpkg", engine=sys.argv[1]))
peak = [l for l in open("/proc/self/status") if l.startswith("VmHWM")][0].split()[1]
print(f"{sys.argv[1]:8} {n:,} features, peak {int(peak) / 1024:,.0f} MB")
"""
for mode in ("stream", "pyogrio", "fiona"):
print(subprocess.run([sys.executable, "-c", PROBE, mode], capture_output=True, text=True).stdout.strip())stream 338,611 features, peak 190 MB
pyogrio 338,611 features, peak 755 MB
fiona 338,611 features, peak 877 MB
Where the C-to-Python boundary sits
Both libraries are thin wrappers over the same GDAL vector API, so they can read exactly the same formats and produce exactly the same geometry. What differs is the boundary between C and Python, and where that boundary sits determines everything else.
Fiona was designed around Python idioms: a dataset is a context manager, features are dicts, iteration is lazy. That is elegant and, for streaming work, exactly right — memory does not grow with file size, and you can process a 40 GB GeoJSON on a laptop. The cost is that every feature crosses the C boundary individually and becomes several Python objects, which dominates the runtime for bulk reads.
pyogrio was built for the opposite case: you want the whole thing as a DataFrame. It reads columns in bulk, hands NumPy arrays back, and lets Shapely 2's vectorised constructors turn a WKB array into a GeometryArray in one call. Because the transfer is columnar, it can also push work down into GDAL — selecting columns, filtering rows, skipping features — so the excluded data never materialises anywhere.
That pushdown matters more than raw speed. where="INSPIREID > 50000000" is evaluated by the driver, and only matching rows are transferred. Fiona can do the same — GeoPandas passes where, columns and bbox through to its filtering — but pyogrio's own arguments for windows and SQL have no Fiona counterpart, and are dropped rather than refused.
GeoPandas 1.0 made pyogrio the default, which is the right choice for the common case but does mean upgrades can shift behaviour: dtypes, datetime parsing, and null handling all have small differences. For a pipeline whose output feeds something downstream, setting gpd.options.io_engine explicitly and recording the library versions in the run metadata costs nothing and removes an entire class of "it changed and nobody touched it" incidents.
Arguments one engine ignores
- pyogrio is the default from GeoPandas 1.0: Old code that relied on Fiona's dtype behaviour may see differences after an upgrade.
skip_features=,max_features=andsql=are pyogrio-only, and Fiona ignores them silently: userows=slice(...)for a window that both engines honour.skip_featuresis cheap only on indexed formats: on Rutland's 21,253 parcels, skipping 20,000 took 10 ms from a GeoPackage and 551 ms from GeoJSON, which is parsed in full every time.use_arrow=Trueneeds pyarrow and a recent GDAL: It is the fastest path; where columns have missing values its nullable dtypes can differ from the default read, so comparedtypeswhen switching.- Fiona 1.9+ changed its Feature objects: They are now class instances rather than plain dicts, though still mapping-like.
- Both need the driver: Neither can read a format GDAL was not compiled with. Check
pyogrio.list_drivers(). - Writes differ subtly:
promote_to_multi, encoding defaults and layer options are not identical. Test the round trip when switching.
Internal links
- What GDAL and OGR Actually Are (and Why Everything Depends on Them)
- How to Speed Up GeoPandas: Tips for Large Datasets
- How to Process a Very Large GeoPackage in Chunks with Python
- GeoPandas Installation Fails: How to Fix Common Errors
- GeoPandas Basics: Working with Spatial Data in Python
- Fixing Memory Errors in GeoPandas When Working with Large Files
FAQ
Which engine does GeoPandas use by default?
pyogrio, since GeoPandas 1.0. Earlier versions used Fiona. Set gpd.options.io_engine or pass engine= per call to be explicit.
Is pyogrio always faster?
For bulk reads and writes, substantially — 8.7 times faster on the 338,611 Birmingham parcels, 1.72 s against 15.02. Fiona wins when you want to stream features without building a frame at all, because its memory use does not grow with file size.
Why does columns= raise a TypeError?
With GeoPandas 1.x it should not: columns= works with both engines. A TypeError there points to an old GeoPandas. The arguments to watch are skip_features=, max_features= and sql=, which Fiona ignores without raising anything.
Should I remove Fiona from my environment?
No. It is still the right tool for streaming, and some libraries depend on it. Keeping both installed costs nothing.
Will switching engines change my data?
Usually not the geometry, but possibly the dtypes — nullable integers, datetime parsing and null handling differ slightly. Compare dtypes after switching, and pin the engine in production.
What does use_arrow=True do?
It uses GDAL's Arrow stream interface, avoiding a conversion step and returning Arrow-backed nullable dtypes. It is the fastest option where pyarrow and a recent GDAL are available.
Which should I use for writing?
pyogrio, for the same speed reasons, plus conveniences like promote_to_multi. Fiona is preferable when you are copying a schema verbatim from one dataset to another.