How to Cache Spatial Data in a Map App

Problem statement

Caching is the difference between a map app that responds and one that does not. In Streamlit's model the whole script re-runs on every interaction, so without caching each widget change re-reads the file, re-projects it and rebuilds every derived object.

Measured on a 15 MB shapefile:

                  first run    reruns
uncached             0.77 s     0.23–0.25 s
@st.cache_data       0.43 s     0.04–0.06 s

Four to five times, from one decorator — and the ratio grows with the file. On a 500 MB layer the uncached version is unusable and the cached one is fine.

The complications are all in the details: which decorator, what the key is, what happens to a GeoDataFrame that cannot be hashed, and how to stop a cache growing until the worker is killed.

Quick answer

import streamlit as st


@st.cache_data(ttl=3600, max_entries=8, show_spinner="Loading…")
def load_layer(path: str, simplify_deg: float = 0.001):
    """Values — DataFrames, GeoDataFrames, arrays. Returns a copy."""
    import geopandas as gpd
    gdf = gpd.read_file(path).to_crs(4326)
    gdf["geometry"] = gdf.geometry.simplify(simplify_deg, preserve_topology=True)
    return gdf


@st.cache_resource
def connection(path: str):
    """Connections, models, file handles. Returns the same object to everyone."""
    import duckdb
    con = duckdb.connect(path, read_only=True)
    con.execute("load spatial")
    return con


@st.cache_data
def query(_con, region: str, year: int):
    """A leading underscore excludes an argument from the cache key."""
    return _con.execute("select … where region = ? and year = ?",
                        [region, year]).df()

Three rules: values in cache_data, connections in cache_resource, and anything unhashable prefixed with an underscore.

Five things to cache in a spatial app, ordered by value. — Everything below the first line is smaller than the first line.

Step-by-step solution

1. Cache the load, first

The loader is the largest single win in almost every map app, because it runs on every rerun and its result never changes:

@st.cache_data
def load_districts(path):
    return gpd.read_file(path).to_crs(4326)

That one decorator produced the measured 0.23 s → 0.05 s improvement. Everything else on this page is smaller.

2. Cache the derived forms too

A GeoDataFrame is rarely what the map consumes. Converting to GeoJSON, building a lookup, computing a summary — all of these run per rerun unless cached:

@st.cache_data
def as_geojson(_gdf, version: str):
    """Cached separately: the filter changes far more often than the layer."""
    import json
    return json.loads(_gdf.to_json())

Note the version argument. Because _gdf is excluded from the key, something else must identify which layer this is — otherwise two layers share one cache entry.

3. Understand the cache key

cache_data hashes the function's arguments. That means:

A path is a good argument. It is hashable and it identifies the data.
A GeoDataFrame is a bad argument. Streamlit will try to hash it, which is slow at best.
An underscore-prefixed argument is skipped entirely — use it for connections and for large objects, and make sure something else in the signature identifies the result.

A mutable default argument is a trap here as everywhere: it becomes part of the identity of the call in ways that surprise.

4. Choose the right decorator

Return value	Decorator	Why
DataFrame, GeoDataFrame, array, dict	`cache_data`	serialised, copied per caller, safe to mutate
database connection	`cache_resource`	cannot be serialised; must be shared
ML model, tokeniser	`cache_resource`	expensive, large, read-only
an open file handle	`cache_resource`	not copyable

Using cache_data on a connection raises an unpicklable-object error. Using cache_resource on a DataFrame gives every session the same object, so one session's mutation changes everybody's.

5. Bound the cache

An unbounded cache is a memory leak with a friendly name. A parameterised loader called with many different arguments accumulates one entry per combination:

@st.cache_data(ttl=3600, max_entries=8)
def load_year(year: int):
    ...

max_entries evicts the least recently used; ttl expires by age. For a map app the important one is usually max_entries, because the parameter space is small and the entries are large.

6. Cache the query, not the data, when the data is large

For anything that does not fit comfortably in the app, keep the data in a database and cache the result:

@st.cache_data(max_entries=32)
def query(_con, region, year, min_rate):
    return _con.execute(SQL, [region, year, min_rate]).df()

Each cache entry is then the size of a filtered result rather than of the whole dataset — which is what makes the app's memory a function of what is displayed.

Checklist of four cache-key rules and one anti-pattern. — The file-version rule is what stops an app serving the data it read at start-up forever.

Code examples

Example 1 — a caching layer for a spatial app

import json
import streamlit as st
import geopandas as gpd


@st.cache_resource
def duck():
    import duckdb
    con = duckdb.connect("districts.duckdb", read_only=True)
    con.execute("load spatial")
    return con


@st.cache_data(show_spinner="Loading boundaries…", max_entries=4)
def boundaries(path: str, simplify_deg: float = 0.001) -> gpd.GeoDataFrame:
    gdf = gpd.read_file(path).to_crs(4326)
    gdf["geometry"] = gdf.geometry.simplify(simplify_deg, preserve_topology=True)
    return gdf


@st.cache_data(max_entries=16)
def geojson_of(_gdf: gpd.GeoDataFrame, cache_key: str) -> dict:
    """`cache_key` identifies the frame; `_gdf` is excluded from hashing."""
    return json.loads(_gdf.to_json())


@st.cache_data(max_entries=64, ttl=900)
def rates(_con, region: str | None, year: int):
    clauses, params = ["year = ?"], [year]
    if region:
        clauses.append("region = ?")
        params.append(region)
    return _con.execute(
        f"select code, rate from rates where {' and '.join(clauses)}",
        params).df()


def cache_status():
    """Show what is cached — useful in a sidebar during development."""
    st.sidebar.caption(
        "Caches: boundaries ≤4 entries, geojson ≤16, rates ≤64 (15 min TTL)")
    if st.sidebar.button("Clear caches"):
        st.cache_data.clear()
        st.cache_resource.clear()
        st.rerun()

Example 2 — measuring what caching actually buys

from streamlit.testing.v1 import AppTest
import time


def compare_caching(uncached_path, cached_path, runs=3):
    """Run both variants headlessly and print the difference."""
    results = {}
    for label, path in (("uncached", uncached_path), ("cached", cached_path)):
        app = AppTest.from_file(path, default_timeout=120)
        started = time.perf_counter()
        app.run()
        first = time.perf_counter() - started

        times = []
        options = sorted(app.selectbox[0].options)
        for value in options[1:1 + runs]:
            app.selectbox[0].select(value)
            started = time.perf_counter()
            app.run()
            times.append(time.perf_counter() - started)

        results[label] = (first, times)
        print(f"{label:9} first {first:5.2f}s   "
              f"reruns {', '.join(f'{t:.2f}' for t in times)}")

    speedup = (min(results['uncached'][1]) / min(results['cached'][1]))
    print(f"\nreruns are {speedup:.1f}× faster with caching")
    return results

uncached  first  0.77s   reruns 0.25, 0.23, 0.24
cached    first  0.43s   reruns 0.05, 0.04, 0.06

reruns are 4.6× faster with caching

Example 3 — a cache keyed on file modification time

import os
import streamlit as st


def file_version(path: str) -> str:
    """Data that changes on disk should invalidate the cache automatically."""
    stat = os.stat(path)
    return f"{stat.st_mtime_ns}:{stat.st_size}"


@st.cache_data(show_spinner="Loading…")
def load_versioned(path: str, version: str):
    """`version` is part of the key, so a changed file is a cache miss."""
    import geopandas as gpd
    return gpd.read_file(path)


districts = load_versioned("districts.gpkg", file_version("districts.gpkg"))

This is the pattern for data that is refreshed by another process. Without it, the app serves the version it read at start-up until it is restarted — which is one of the most confusing bugs an app can have, because the file on disk is obviously correct.

Explanation

Why `cache_data` returns a copy

If every caller received the same object, one session mutating a DataFrame would change what every other session sees, and the bug would appear as data corruption in an unrelated user's browser.

Serialising and returning a copy prevents that entirely. The cost is a serialisation per call, which for a GeoDataFrame is real but far smaller than re-reading the file — the measured rerun was still 0.04–0.06 s including the copy.

Why the underscore prefix exists

Streamlit hashes arguments to build the cache key, and some arguments cannot be hashed: a database connection, an open file, a very large frame where hashing would dominate the work.

Prefixing the parameter with an underscore excludes it. The responsibility that comes with it is that something else in the signature must identify the result — otherwise two different connections or two different frames map to one entry, and the app serves the wrong data.

Why unbounded caches are a memory problem

Each distinct argument combination is an entry, and entries are the size of what the function returns. A loader parameterised by year with twenty years of data holds twenty layers.

In Streamlit that memory is per server, not per session, so it accumulates as users explore. max_entries is the bound that matters; ttl helps for data that goes stale but does nothing for a cache filling up in one session.

Why caching the query beats caching the data at scale

Holding a large layer in cache_data costs its full size in the server, once. Holding it per session — which is what happens without caching, in session state — costs it per user.

Caching the query result instead keeps the data in the database and each entry proportional to the filtered result. That is the change that makes a 50-million-row dataset workable in an app whose memory footprint is measured in tens of megabytes.

Bar chart of cache memory growing with the number of cached argument combinations. — ttl bounds staleness; max_entries bounds memory. They solve different problems.

Edge cases or notes

cache_data for values, cache_resource for connections. Swapping them gives an unpicklable error or a shared-state bug.
Underscore-prefixed arguments are excluded from the key — make sure something else identifies the result.
max_entries bounds memory; ttl bounds staleness. They solve different problems.
Mutating a cache_data result is safe; mutating a cache_resource object affects everyone.
Include a file version in the key for data refreshed by another process.
st.cache_data.clear() empties everything — useful behind a button in development.
Caches are per server process, so they are duplicated across workers.
Do not cache anything user-specific in cache_resource; it is shared.

Internal links

Reruns and state explained: why your map app redraws everything — why caching is load-bearing
How to build a Streamlit app with an interactive map — where the decorators go
Fixing a map app that takes ten seconds to load — the symptom of no caching
Fixing an app whose memory grows with every user — unbounded caches
How to add filters and widgets that drive a map — caching filtered queries
How to cache GIS pipeline steps — the same idea in a batch job
How to cache downloaded GIS data in Python — caching at the source
How to test a map app without a browser — measuring the improvement

FAQ

Which cache decorator should I use?

@st.cache_data for values such as DataFrames and GeoDataFrames — it returns a copy. @st.cache_resource for database connections and models, which must be shared and cannot be serialised.

How much does caching actually help?

Measured on a 15 MB shapefile, reruns fell from 0.23–0.25 s to 0.04–0.06 s — about 4.6 times — from a single decorator on the loader.

How do I cache a function that takes a database connection?

Prefix the parameter with an underscore — def query(_con, region, year) — so Streamlit skips hashing it, and make sure the other arguments identify the result.

How do I stop the cache growing forever?

Set max_entries. Each distinct argument combination is an entry the size of the return value, and the cache is per server rather than per session.

My data file changed and the app did not notice. Why?

The cache key did not change. Include a file version — modification time and size — as an argument so a new file is a cache miss.

Can I mutate a cached GeoDataFrame?

Yes, if it came from cache_data — you have a copy. A cache_resource object is shared, so mutating it affects every session.

How to Cache Spatial Data in a Map App #

Problem statement #

Quick answer #

Step-by-step solution #

1. Cache the load, first #

2. Cache the derived forms too #

3. Understand the cache key #

4. Choose the right decorator #

5. Bound the cache #

6. Cache the query, not the data, when the data is large #

Code examples #

Example 1 — a caching layer for a spatial app #

Example 2 — measuring what caching actually buys #

Example 3 — a cache keyed on file modification time #

Explanation #

Why cache_data returns a copy #

Why the underscore prefix exists #

Why unbounded caches are a memory problem #

Why caching the query beats caching the data at scale #

Edge cases or notes #

Internal links #

FAQ #

Which cache decorator should I use? #

How much does caching actually help? #

How do I cache a function that takes a database connection? #

How do I stop the cache growing forever? #

My data file changed and the app did not notice. Why? #

Can I mutate a cached GeoDataFrame? #