GeoParquet and Columnar Storage Explained: Why It Reads So Much Faster

On this page

Problem statement

The same 232,717 census blocks — every 2020 census block in Georgia, 17 attribute columns and a polygon each — saved in four formats (the setup is under The data used on this page):

import time
import geopandas as gpd

for path in ["blocks.shp", "blocks.gpkg", "blocks.geojson", "blocks.parquet"]:
    times = []
    for _ in range(2):                                   # the faster of two reads
        t0 = time.perf_counter()
        gdf = gpd.read_parquet(path) if path.endswith("parquet") else gpd.read_file(path)
        times.append(time.perf_counter() - t0)
    print(f"{path:<16} {min(times):>6.2f} s")
blocks.shp         1.73 s
blocks.gpkg        1.77 s
blocks.geojson    15.52 s
blocks.parquet     0.62 s

And then this:

import pandas as pd

t0 = time.perf_counter()
gdf = gpd.read_parquet("blocks.parquet", columns=["GEOID20", "POP20", "geometry"])
print(f"three columns with geometry:    {time.perf_counter() - t0:.2f} s")

t0 = time.perf_counter()
df = pd.read_parquet("blocks.parquet", columns=["GEOID20", "POP20", "HOUSING20"])
print(f"three columns without geometry: {time.perf_counter() - t0:.3f} s")
three columns with geometry:    0.59 s
three columns without geometry: 0.021 s

The second read never touches the geometry column, which is most of the file, so it skips most of the bytes.

That last result is not an optimisation applied to a row-based format — it is impossible in one. The difference is structural, and understanding it tells you when GeoParquet is the right choice and when it is not.

Quick answer

import geopandas as gpd

gdf = gpd.read_file("blocks.gpkg")
gdf.to_parquet("blocks.parquet", compression="zstd")      # write once

gdf = gpd.read_parquet("blocks.parquet")                  # read many times
gdf = gpd.read_parquet("blocks.parquet", columns=["GEOID20", "POP20", "geometry"])
Row-oriented storage compared with columnar for the same table. the layout decides what is cheaprow-orientedcolumnaron diskrecord 1, record 2all ids, all classeswhole records✓one seek!18 seeksone column of 18✕reads all 18✓reads 1compression!mixed types✓one type per runbest for✓editing, lookup✓analysis, scans3 attribute columns of 232,717 blocks: 0.36 s GeoPackage, 0.02 s GeoParquet spatialworkflow.io
Row storage interleaves the columns. Columnar keeps each one contiguous — so reading two of eighteen touches two.
Property GeoPackage GeoParquet
layout row-oriented columnar
read all 232,717 blocks 1.8 s; 1.1 s with use_arrow=True 0.6 s
read 3 attribute columns, no geometry 0.36 s 0.02 s
file size 360 MB 263 MB; 157 MB zstd and Hilbert-sorted
bbox read, 10 km of central Atlanta 0.04 s, R-tree 0.06 s sorted; 0.25 s unsorted
random access by id fast (indexed) slow
append a row cheap rewrite a file
editable in QGIS yes export only

All from the runs on this page. GeoParquet's advantage on a full read is real but modest for polygon-heavy data, because geometry is nearly all the bytes and every format has to turn it into Shapely objects. It is overwhelming when you can leave the geometry out.

Use GeoParquet for analytical reads and GeoPackage for editing and interchange. They are not competitors; they answer different questions.

What columnar means on disk, and writing it properly

1. Understand what "columnar" means physically

A table has to be laid out linearly on disk. There are two ways to do it.

Row-oriented — everything about record 1, then everything about record 2:

[GEOID20=130010001001000|UR20=R|POP20=23|geom=...] [GEOID20=130010001001001|UR20=R|POP20=0|geom=...] ...

Reading POP20 alone means seeking past every other field of every row, or more realistically reading whole pages and discarding most of them. Shapefile, GeoPackage and most databases work this way, and it is the right choice when you routinely want whole records.

Column-oriented — every GEOID20, then every UR20, then every POP20:

[GEOID20: 130010001001000, 130010001001001, ...] [UR20: R, R, U, ...] [POP20: 23, 0, 41, ...] [geom: WKB, ...]

Reading POP20 alone means reading one contiguous run of bytes and nothing else. That is the whole trick, and everything else follows from it.

2. Follow the consequences

The four advantages that follow from contiguous column storage. one layout decision, four winscontiguous columnsthe decisioneverything below followscolumn pruningread 3 of 18skip the geometry: 0.02 sbetter compressionone type per runcounty code: < 1 byte a rowrow-group statisticsmin and maxskip whole groupstyped, vectorised readsno parsingmemcpy, not atofGeometry is the exception — WKB barely compresses spatialworkflow.io
One layout decision produces four separate wins.

Column pruning. Read only the columns you name — as long as geometry is not one of them, which is the catch measured above.

Better compression. A column holds one type with repetitive values. A column of county codes drawn from Georgia's 159 counties compresses to almost nothing by dictionary encoding; in a row layout those values are separated by unrelated bytes, so the compressor cannot see the pattern. The file's own metadata shows it:

import pyarrow.parquet as pq

meta = pq.read_metadata("blocks.parquet")
rg = meta.row_group(0)
print(f"{meta.num_row_groups} row group(s), {rg.num_rows:,} rows")
raw = {"COUNTYFP20": 3, "UR20": 1, "GEOID20": 15, "INTPTLAT20": 11}   # characters per value as text
for j in range(rg.num_columns):
    col = rg.column(j)
    name = col.path_in_schema
    if name in ("COUNTYFP20", "UR20", "GEOID20", "INTPTLAT20", "POP20", "geometry"):
        per_row = col.total_compressed_size / rg.num_rows
        text = f"   (as text: {raw[name]} bytes)" if name in raw else ""
        print(f"  {name:<11} {col.total_compressed_size / 1e6:>6.1f} MB  {per_row:>8.2f} bytes a row{text}")
1 row group(s), 232,717 rows
  COUNTYFP20     0.2 MB      0.88 bytes a row   (as text: 3 bytes)
  GEOID20        1.9 MB      8.11 bytes a row   (as text: 15 bytes)
  UR20           0.0 MB      0.15 bytes a row   (as text: 1 bytes)
  INTPTLAT20     2.0 MB      8.73 bytes a row   (as text: 11 bytes)
  POP20          0.3 MB      1.35 bytes a row
  geometry     251.8 MB   1082.07 bytes a row

The county code is three characters as text and under a byte a row here, because dictionary encoding stores each of the 159 codes once and then a small index per row, and run-length encoding collapses runs of the same index. Pyarrow applies that encoding before the compressor, so the metadata's own "compression ratio" for these columns reads close to 1× — the saving has already happened. geometry is the opposite case: WKB is already dense binary, it barely compresses, and it is nearly all of the file.

Predicate pushdown. Each row group stores per-column min and max, so a reader can skip an entire group without decompressing it:

rural = gpd.read_parquet("blocks.parquet", filters=[("UR20", "==", "R")])
print(f"{len(rural):,} rural blocks")
122,668 rural blocks

Typed, vectorised reads. A column of int64 is already a contiguous typed array, so it becomes a NumPy array with a memory copy rather than a per-value parse.

3. Write GeoParquet properly

blocks.to_parquet(
    "blocks_zstd.parquet",
    compression="zstd",            # smaller than the snappy default
    compression_level=3,
    row_group_size=10_000,         # rows per group — the pushdown granularity
    write_covering_bbox=True,      # per-row bbox columns for spatial filtering
    geometry_encoding="WKB",       # the interoperable default
    index=False,
)
import os
for path in ["blocks.parquet", "blocks_zstd.parquet"]:
    print(f"{path:<20} {os.path.getsize(path) / 1e6:>6.0f} MB, "
          f"{pq.read_metadata(path).num_row_groups} row groups")
blocks.parquet          263 MB, 1 row groups
blocks_zstd.parquet     241 MB, 24 row groups
Option Effect
compression="zstd" smaller than snappy — measured above; "snappy" is the default and the most widely readable
row_group_size smaller groups mean finer filtering and more metadata overhead
write_covering_bbox=True adds a bbox struct column so bbox= reads can skip groups
geometry_encoding "WKB" is portable; "geoarrow" is faster and less widely supported
index=False do not persist a meaningless RangeIndex as a column

Row-group size is the one real tuning decision, and the default is usually wrong for spatial filtering: pyarrow puts up to about a million rows in a group, so this whole file was written as a single group, and no statistic can skip part of one group. Pick a size that gives tens of groups or more — 10,000 rows here, 24 groups.

4. Use spatial filtering

Parquet has no R-tree, but bounding-box statistics per row group achieve much of the same. What decides whether they help is the order the rows are in — so write three copies: shuffled, as the census file came (sorted by block code), and sorted along a Hilbert curve:

import numpy as np


def groups_touched(path, bbox):
    """How many row groups a bbox read cannot rule out, from the bbox column statistics."""
    meta, (xmin, ymin, xmax, ymax), n = pq.read_metadata(path), bbox, 0
    names = [meta.schema.column(j).path for j in range(meta.num_columns)]
    for i in range(meta.num_row_groups):
        st = {names[j]: meta.row_group(i).column(j).statistics for j in range(meta.num_columns)}
        n += not (st["bbox.xmin"].min > xmax or st["bbox.xmax"].max < xmin or
                  st["bbox.ymin"].min > ymax or st["bbox.ymax"].max < ymin)
    return n


orders = {
    "shuffled": blocks.sample(frac=1, random_state=0),
    "census order": blocks,
    "hilbert": blocks.iloc[blocks.hilbert_distance().argsort()],
}
atlanta = (1_060_000, 1_247_000, 1_070_000, 1_257_000)      # 10 km square, central Atlanta
for name, gdf in orders.items():
    path = f"blocks_{name.replace(' ', '_')}.parquet"
    gdf.to_parquet(path, row_group_size=10_000, write_covering_bbox=True, index=False)
    t0 = time.perf_counter()
    subset = gpd.read_parquet(path, bbox=atlanta)
    print(f"{name:<13} {time.perf_counter() - t0:.3f} s  {len(subset):,} blocks  "
          f"{groups_touched(path, atlanta):>2} of {pq.read_metadata(path).num_row_groups} row groups read")
shuffled      0.246 s  3,179 blocks  24 of 24 row groups read
census order  0.266 s  3,179 blocks  24 of 24 row groups read
hilbert       0.062 s  3,179 blocks   5 of 24 row groups read

The saving depends entirely on spatial locality within row groups. If the data is written in a spatially coherent order, a bounding box intersects few groups and most of the file is skipped. If the rows are in random order, every group's bbox covers the whole state and nothing can be skipped. The order the census file arrives in turned out to be no better than shuffled for this query: each 10,000-row group still spans enough of the state that none could be ruled out. Only the Hilbert sort let the reader skip most of the file.

The sort pays a second time, in file size. Neighbouring census blocks share their boundaries, so in Hilbert order the same coordinates recur a few rows apart — close enough for zstd, which looks back over a long window, to store them once:

for name, gdf in orders.items():
    for codec in ["snappy", "zstd"]:
        gdf.to_parquet("size_test.parquet", compression=codec, row_group_size=10_000, index=False)
        print(f"{name:<13} {codec:<7} {os.path.getsize('size_test.parquet') / 1e6:>4.0f} MB")
shuffled      snappy   266 MB
shuffled      zstd     238 MB
census order  snappy   265 MB
census order  zstd     237 MB
hilbert       snappy   241 MB
hilbert       zstd     157 MB

hilbert_distance() maps 2-D position to a 1-D ordering that preserves proximity, so spatially close features land in the same row group. It is a one-off cost at write time and it is what makes bbox= reads worthwhile. Note it returns a Series of distances to sort by — gdf.sort_values(by=gdf.hilbert_distance()) raises, because by= takes column names; iloc[... .argsort()] is the working form.

GeoParquet file size of Georgia’s census blocks by row order and codec: zstd on Hilbert-sorted rows is a third smaller than any other combination. the same rows, the same codec — the order changes the sizecensus order, snappy265 MBshuffled, zstd238 MBcensus order, zstd237 MBHilbert, snappy241 MBHilbert, zstd157 MB0100 MB200 MB300 MBNeighbouring blocks share boundaries; sorted together, zstd finds the repeats spatialworkflow.io
Sorting is usually sold on faster bbox reads. On polygon coverages it shrinks the file too.

5. Know when GeoParquet is the wrong choice

Parquet is immutable and analytical. It has no support for:

  • Updating a row. You rewrite the file, or write a new one.
  • Appending efficiently. Appending means a new file; readers then treat a directory of files as one dataset.
  • Random access by key. No index — finding one id means scanning, or relying on statistics.
  • Editing in a desktop GIS. QGIS reads GeoParquet through GDAL's Parquet driver but is not an editing environment for it.
  • Multiple layers in one file. One Parquet file is one table.
# ❌ this rewrites the whole file to change one value
gdf.loc[gdf["GEOID20"] == "130010001001000", "POP20"] = 24
gdf.to_parquet("blocks.parquet")

# ✅ mutable data belongs in GeoPackage or PostGIS

The practical pattern is GeoPackage or PostGIS as the source of record, GeoParquet as the analytical copy — written once when the source changes, read many times.

Georgia's census blocks as GeoParquet against row formats

The two arguments that decide what a GeoParquet file costs and what it can skip. the two arguments worth settinggdf.to_parquet(path,compression="zstd",compression_level=3,write_covering_bbox=True)①②①Smaller than snappy at a similar read speed. Snappy is the default becauseit is the fastest to write, not the smallest.②Stores a bounding box column, so a reader can skip row groups that falloutside a query window instead of reading the file.Columnar storage only pays off if the reader is allowed to skip something spatialworkflow.io
The comparison below measures both against GeoPackage and GeoJSON.

The data used on this page

Every 2020 census block in Georgia, from the US Census Bureau's TIGER/Line files: 232,717 polygons with 17 attribute columns, reprojected to the Albers equal-area projection for the contiguous US (EPSG:5070) so coordinates and areas are in metres. The setup writes the four copies the problem statement reads.

import os
import time

import geopandas as gpd
import requests

URL = "https://www2.census.gov/geo/tiger/TIGER2020/TABBLOCK20/tl_2020_13_tabblock20.zip"
with open("tl_2020_13_tabblock20.zip", "wb") as f:
    f.write(requests.get(URL, timeout=600).content)
blocks = gpd.read_file("tl_2020_13_tabblock20.zip").to_crs(5070)

blocks.to_file("blocks.shp")
blocks.to_file("blocks.gpkg")
blocks.to_file("blocks.geojson")
blocks.to_parquet("blocks.parquet")
print(f"{len(blocks):,} blocks, {len(blocks.columns)} columns, EPSG:{blocks.crs.to_epsg()}")
232,717 blocks, 18 columns, EPSG:5070

Example 1: measuring the difference on your own data

import os
import time
from pathlib import Path

import geopandas as gpd


def compare_formats(gdf, out_dir="fmt_test", columns=None):
    out = Path(out_dir)
    out.mkdir(exist_ok=True)
    columns = columns or [c for c in gdf.columns if c != gdf.geometry.name][:3]

    targets = [
        ("GeoPackage", out / "t.gpkg", lambda p: gdf.to_file(p, driver="GPKG")),
        ("GeoJSON", out / "t.geojson", lambda p: gdf.to_file(p, driver="GeoJSON")),
        ("Parquet snappy", out / "t_snappy.parquet",
         lambda p: gdf.to_parquet(p, compression="snappy", write_covering_bbox=True, row_group_size=10_000)),
        ("Parquet zstd", out / "t_zstd.parquet",
         lambda p: gdf.to_parquet(p, compression="zstd", compression_level=3,
                                  write_covering_bbox=True, row_group_size=10_000)),
        ("FlatGeobuf", out / "t.fgb", lambda p: gdf.to_file(p, driver="FlatGeobuf")),
    ]

    print(f"{len(gdf):,} rows, {len(gdf.columns)} columns\n")
    print(f"{'format':<16}{'write':>9}{'size':>10}{'read all':>10}"
          f"{'read 3 cols':>13}{'bbox read':>11}")

    minx, miny, maxx, maxy = gdf.total_bounds
    dx, dy = (maxx - minx) * 0.1, (maxy - miny) * 0.1
    bbox = (minx + dx, miny + dy, minx + 3 * dx, miny + 3 * dy)

    for name, path, write in targets:
        t0 = time.perf_counter(); write(path); w = time.perf_counter() - t0
        size = os.path.getsize(path) / 1e6

        is_pq = path.suffix == ".parquet"
        reader = gpd.read_parquet if is_pq else gpd.read_file

        t0 = time.perf_counter(); reader(path); r_all = time.perf_counter() - t0
        t0 = time.perf_counter(); reader(path, columns=[*columns, "geometry"] if is_pq else columns)
        r_cols = time.perf_counter() - t0
        t0 = time.perf_counter(); reader(path, bbox=bbox)
        r_bbox = time.perf_counter() - t0

        print(f"{name:<16}{w:>8.2f}s{size:>9.0f}M{r_all:>9.2f}s{r_cols:>12.2f}s{r_bbox:>10.2f}s")


compare_formats(blocks)
232,717 rows, 18 columns

format              write      size  read all  read 3 cols  bbox read
GeoPackage          2.19s      360M     1.67s        0.95s      0.04s
GeoJSON            23.80s      940M    14.77s       14.50s     13.36s
Parquet snappy      1.22s      274M     0.73s        0.67s      0.28s
Parquet zstd        1.71s      241M     0.72s        0.64s      0.27s
FlatGeobuf          2.62s      333M     1.65s        0.96s      0.04s

Four readings worth pulling out. GeoJSON is an order of magnitude slower at everything, and naming columns or a bbox does not help it, because the whole file has to be parsed either way. Parquet reads everything fastest, but only by two to three times here, because geometry dominates. GeoPackage and FlatGeobuf have real spatial indexes and win the bbox read outright — hundredths of a second, against a quarter of a second for Parquet written in the census file's order — so for a workload that is only ever spatial filtering, they beat an unsorted Parquet file. And Parquet writes fastest, so the convert-once cost is small.

Much of GeoPackage's gap is GDAL's default read path rather than the format. Reading through Arrow, and leaving the geometry out, change the picture:

for use_arrow in (False, True):
    t0 = time.perf_counter()
    gpd.read_file("blocks.gpkg", use_arrow=use_arrow)
    print(f"GeoPackage, use_arrow={use_arrow!s:<5}         {time.perf_counter() - t0:.2f} s")
t0 = time.perf_counter()
gpd.read_file("blocks.gpkg", columns=["GEOID20", "POP20", "HOUSING20"], ignore_geometry=True)
print(f"GeoPackage, 3 columns, no geometry   {time.perf_counter() - t0:.2f} s")
t0 = time.perf_counter()
pd.read_parquet("blocks.parquet", columns=["GEOID20", "POP20", "HOUSING20"])
print(f"GeoParquet, 3 columns, no geometry   {time.perf_counter() - t0:.3f} s")
GeoPackage, use_arrow=False         2.24 s
GeoPackage, use_arrow=True          1.14 s
GeoPackage, 3 columns, no geometry   0.36 s
GeoParquet, 3 columns, no geometry   0.014 s

With Arrow, GeoPackage's full read roughly halves, closing most of the gap to GeoParquet. The attribute-only read is where the layouts part company: GeoPackage still walks every row to pick the three fields out, and Parquet reads three contiguous columns.

Example 2: a conversion utility with sensible defaults

from pathlib import Path
import time
import geopandas as gpd


def to_geoparquet(src, dst=None, *, sort_spatially=True, row_group_size=10_000,
                  compression="zstd", drop_columns=None, target_crs=None):
    """Convert any GeoPandas-readable file to a well-formed GeoParquet."""
    src = Path(src)
    dst = Path(dst) if dst else src.with_suffix(".parquet")

    t0 = time.perf_counter()
    gdf = gpd.read_file(src)
    read_s = time.perf_counter() - t0

    if drop_columns:
        gdf = gdf.drop(columns=[c for c in drop_columns if c in gdf.columns])
    if target_crs and gdf.crs != target_crs:
        gdf = gdf.to_crs(target_crs)
    if gdf.crs is None:
        raise ValueError(f"{src} has no CRS — GeoParquet requires one in its metadata")

    if sort_spatially and len(gdf) > row_group_size:
        t0 = time.perf_counter()
        gdf = gdf.iloc[gdf.hilbert_distance().argsort()].reset_index(drop=True)
        sort_s = time.perf_counter() - t0
    else:
        sort_s = 0.0

    t0 = time.perf_counter()
    tmp = dst.with_suffix(".tmp.parquet")
    gdf.to_parquet(tmp, compression=compression, compression_level=3,
                   row_group_size=row_group_size, write_covering_bbox=True,
                   index=False)
    tmp.replace(dst)
    write_s = time.perf_counter() - t0

    src_mb = src.stat().st_size / 1e6
    dst_mb = dst.stat().st_size / 1e6
    print(f"{src.name} → {dst.name}")
    print(f"  {len(gdf):,} rows, {len(gdf.columns)} columns, {gdf.crs}")
    print(f"  read {read_s:.1f}s  sort {sort_s:.1f}s  write {write_s:.1f}s")
    print(f"  {src_mb:,.0f} MB → {dst_mb:,.0f} MB ({100 * (1 - dst_mb/src_mb):.0f}% smaller)")
    return dst


to_geoparquet("blocks.gpkg", "blocks_clean.parquet", target_crs=5070,
              drop_columns=["GEOIDFQ20", "INTPTLAT20", "INTPTLON20"])
blocks.gpkg → blocks_clean.parquet
  232,717 rows, 15 columns, EPSG:5070
  read 1.8s  sort 0.1s  write 1.8s
  360 MB → 145 MB (60% smaller)

The Hilbert sort is a one-off cost and is what makes every later bbox= read fast. Without it, spatially adjacent features can be scattered across row groups and the bbox statistics cannot exclude anything.

Requiring a CRS is deliberate: GeoParquet stores it in the file's metadata, and a file written without one is valid Parquet but not valid GeoParquet — readers then have no way to interpret the coordinates.

The temp-and-rename means an interrupted conversion leaves no half-written file to be mistaken for a complete one.

Example 3: reading larger-than-memory data

Parquet's structure allows things a single-file read cannot:

from pathlib import Path

import pyarrow as pa
import pyarrow.parquet as pq
import pyarrow.dataset as ds
import geopandas as gpd

# ── (a) one row group at a time ─────────────────────────────────────────────
def iter_row_groups(path, columns=None):
    pf = pq.ParquetFile(path)
    for i in range(pf.num_row_groups):
        table = pf.read_row_group(i, columns=columns)
        yield gpd.GeoDataFrame.from_arrow(table)

total_area, rows = 0.0, 0
for chunk in iter_row_groups("blocks_clean.parquet", columns=["GEOID20", "geometry"]):
    total_area += chunk.area.sum()
    rows += len(chunk)
print(f"{rows:,} rows, {total_area / 1e6:,.0f} km², peak memory bounded by one group")

# ── (b) partitioned writing, so future reads skip whole directories ─────────
# GeoDataFrame.to_parquet writes one file, so write one per county yourself
for county, part in blocks.groupby("COUNTYFP20"):
    folder = Path(f"blocks_by_county/COUNTYFP20={county}")
    folder.mkdir(parents=True, exist_ok=True)
    part.drop(columns="COUNTYFP20").to_parquet(folder / "part-0.parquet", compression="zstd",
                                               write_covering_bbox=True)

# ── (c) a directory of files as one dataset ─────────────────────────────────
# declare the partition column a string, or "121" is read back as the integer 121
by_county = ds.partitioning(pa.schema([("COUNTYFP20", pa.string())]), flavor="hive")
fulton = gpd.read_parquet("blocks_by_county/", partitioning=by_county,
                          filters=[("COUNTYFP20", "==", "121"), ("UR20", "==", "U")])   # Fulton County, urban
files = ds.dataset("blocks_by_county/", format="parquet", partitioning=by_county).files
print(f"{len(fulton):,} rows read from a {len(files)}-file dataset")
232,717 rows, 153,905 km², peak memory bounded by one group
11,384 rows read from a 159-file dataset

Two things in (b) and (c) are not what the pyarrow documentation leads you to expect. GeoDataFrame.to_parquet has no partition_cols — passing one fails with IsADirectoryError — so the loop writes the Hive layout itself. And Hive partitioning infers the type of the directory values, so COUNTYFP20=121 comes back as int32, the string filter fails with ArrowNotImplementedError: Function 'equal' has no kernel matching input types (int32, string), and county 001 would silently become 1. Declaring the partition schema fixes both.

Partitioning by a column writes blocks_by_county/COUNTYFP20=121/part-0.parquet, and a filter on COUNTYFP20 then skips entire directories without opening a file. That is a different mechanism from row-group statistics and much more decisive, but it only helps for the column you partitioned on — so partition by whatever you filter on most.

Per-row-group iteration is the memory-bounded read: peak usage is one group, so a dataset far larger than memory is no harder than a small one. The same reasoning as processing a large GeoPackage in chunks, with the advantage that the chunk boundaries are already in the file.

Physical layout explains every difference

Spatially sorted row groups compared with unsorted ones for bbox filtering. statistics can only skip what they can excludeHilbert-sorted — 5 of 24 readeach group is compactunsorted — 24 of 24 readevery group spans everythingSort by hilbert_distance() before writing spatialworkflow.io
Row-group statistics can only skip what they can exclude. Unsorted rows make every group cover everything.

The performance difference between GeoParquet and a row-based format is not an implementation detail — it follows from a decision about physical layout, and every advantage and limitation traces back to it.

Storage is linear; a table is two-dimensional. Something has to be chosen, and the choice determines what is cheap. Row-oriented layout keeps a record's fields adjacent, so reading or writing a whole record touches one contiguous region — excellent for transactional work, where you fetch a record by key, edit it and write it back. Column-oriented layout keeps a column's values adjacent, so reading one column touches one contiguous region — excellent for analytical work, where you compute over a few columns across many rows.

Compression is where the difference compounds, and it is not obvious in advance. Compressors work by finding repetition. Georgia's 232,717 county codes are drawn from 159 values, and dictionary encoding reduces them to a lookup table plus a small index per row — under a byte a row, measured earlier, against three as text. The same values in a row layout are separated by unrelated bytes of other columns, so the pattern is invisible within any compression window. Columnar storage does not merely compress better; it makes a different class of encoding possible.

Parsing cost disappears for typed columns. A float64 column in Parquet is already a contiguous run of IEEE doubles, so becoming a NumPy array is a memory copy. In GeoJSON the same value is the text "-2.24261084" and must be converted digit by digit — which is why GeoJSON read about 24 times slower than Parquet on the blocks while being under four times the size: the gap is mostly parsing, not bytes.

How well each kind of column compresses, and why geometry is the exception. the saving is not spread evenlycolumncompressesbecausea repeated categoryvery welldictionary, then run-lengthinteger countswell1.35 bytes a row, not 8WKB geometrybarelyalready dense binaryso a geometry-heavy layer saves less than a benchmark run on attribute tables suggestsCompressors work on runs of similar bytes, and a column is the longest such run available spatialworkflow.io
Which is the next paragraph: geometry is where the columnar advantage runs out.

Geometry is the exception, and it is worth knowing. WKB is already dense binary, so it barely compresses and it still has to be parsed into Shapely objects on read. In the measurements above, geometry is 252 MB of the 263 MB file — 1,082 bytes a row, against under a byte for the county code. So GeoParquet's advantage is largest on attribute-heavy data and smallest on geometry-heavy data with few attributes — and the columns= win is largest precisely when you can leave the geometry column out entirely.

The absence of a spatial index is the honest trade-off. GeoPackage carries an R-tree, so a bbox query goes straight to the matching rows regardless of storage order. Parquet has only per-row-group min/max statistics, which can exclude a group but cannot locate a row. This works well when the data is spatially sorted — a Hilbert ordering makes each group cover a compact region, so a small bbox intersects few groups — and not at all when rows are in arbitrary order, since then every group's bbox spans the whole extent. That is why the sort is not an optional refinement but the thing that makes spatial filtering work.

Finally, immutability is a design decision rather than a missing feature. Parquet's compression and encoding are computed per row group over the whole group, so changing one value requires re-encoding the group and rewriting the file. That is what buys the compression ratio and the read speed. It also means Parquet is the wrong home for data people edit — the natural pattern is a mutable source of record in GeoPackage or PostGIS, with GeoParquet as a derived, regenerated analytical copy.

CRS metadata, column skipping and spatial sorting

  • GeoParquet requires a CRS in its metadata. Writing a frame with crs=None produces valid Parquet that is not valid GeoParquet.
  • gpd.read_parquet accepts columns=; gpd.read_file does too but cannot skip them physically.
  • bbox= only helps if the data is spatially sorted. Use hilbert_distance() before writing.
  • write_covering_bbox=True adds bbox columns and is what makes bbox= reads possible at all.
  • Row-group size is the main tuning knob. Pyarrow's default puts up to about a million rows in one group, which leaves nothing for statistics to skip in a file this size. Aim for tens of groups or more.
  • zstd was about 10% smaller than snappy on the blocks in their original order and 35% smaller once Hilbert-sorted; snappy has the widest reader support.
  • geometry_encoding="geoarrow" is faster than WKB and less widely readable. Prefer WKB for interchange.
  • Partitioning by a column lets readers skip whole directories, which beats row-group statistics — but only for that column.
  • QGIS reads GeoParquet when its GDAL has the Parquet driver (GDAL 3.5 or later), and can export to it; it cannot edit a Parquet file in place.
  • A directory of Parquet files is one dataset to pyarrow.dataset, which is how appends are normally handled.

FAQ

Why is GeoParquet so much faster to read?

It only reads the columns you ask for, its columnar layout compresses attribute columns far better, and typed columns become NumPy arrays by memory copy rather than by parsing text. For polygon-heavy data the full-read gain is modest — geometry is most of the bytes in any format — and the large gains come from reading attributes without geometry.

Should I replace GeoPackage with GeoParquet?

Not replace — complement. GeoPackage for editing, interchange and desktop GIS; GeoParquet as a derived analytical copy that is regenerated when the source changes.

Does GeoParquet support spatial indexing?

Not an R-tree. It stores per-row-group bounding boxes, which can skip groups. That only works if the data is spatially sorted, so sort by hilbert_distance() before writing.

What row-group size should I use?

Enough rows that per-group metadata is negligible, few enough that the file has tens of groups or more — 10,000 gave 24 groups for Georgia's 232,717 blocks. Smaller gives finer filtering and more metadata overhead; larger gives better compression and coarser skipping. The pyarrow default of about a million rows is too coarse for spatial filtering on most single files.

Can I append to a GeoParquet file?

Not to one file. Write a new file into the same directory and read the directory as a dataset with pyarrow.dataset, which is the standard pattern.

Which compression should I choose?

zstd for the best size, snappy for the widest reader compatibility. Neither changes the data; both are lossless. zstd gains most on spatially sorted data, where shared boundaries repeat nearby.

Can QGIS open GeoParquet?

Yes, when its GDAL (3.5 or later) is built with the Arrow/Parquet driver, and it can export to GeoParquet through the same driver. A Parquet file cannot be edited in place, though, which is another reason to keep an editable GeoPackage alongside.