How to Render a Million Points in a Map App with pydeck #

Problem statement #

folium stops being usable somewhere around ten thousand features — measured, 10,000 CircleMarkers is 4.78 MB of HTML and 1.67 s to build, and 50,000 is 23.88 MB and 8.68 s. It builds a DOM object per feature, and browsers do not enjoy fifty thousand of those.

pydeck wraps deck.gl, which draws with WebGL. It handles hundreds of thousands of points comfortably, and it has its own ceiling that arrives in a different place:

pydeck points     HTML page    build time
        1,000       0.08 MB       0.00 s
       10,000       0.74 MB       0.01 s
      100,000       7.39 MB       0.06 s
    1,000,000      73.78 MB       0.64 s

The page embeds the data as JSON, so a million points is a 74 MB HTML file. The rendering is not the problem; the transport is. Getting past it means changing how the data reaches the browser, not how it is drawn.

Quick answer #

import pydeck as pdk
import streamlit as st

layer = pdk.Layer(
    "ScatterplotLayer",
    data=points[["lon", "lat", "value"]],      # only the columns you use
    get_position=["lon", "lat"],
    get_radius=30,
    get_fill_color=[14, 165, 233, 140],
    radius_min_pixels=1,
    radius_max_pixels=8,
    pickable=True,
)

deck = pdk.Deck(
    layers=[layer],
    initial_view_state=st.session_state.view,
    tooltip={"text": "{value}"},
    map_style="light",
)
st.pydeck_chart(deck, use_container_width=True)

Three details that matter more than the layer type: send only the columns you use, keep the view state in session_state, and set radius_min_pixels so points remain visible when zoomed out.

Step-by-step solution #

1. Choose the layer from what the data means #

The aggregating layers are the ones that change the ceiling: ScreenGridLayer and HexagonLayer bin on the GPU, so a million points becomes a few thousand drawn cells.

2. Send only the columns the layer reads #

pydeck serialises the DataFrame you hand it into the page. Every column travels, whether the layer uses it or not.

layer_data = points[["lon", "lat", "value"]]      # not points

On a million rows, dropping five unused columns is tens of megabytes of page.

3. Round the coordinates #

Six decimal places is about 11 cm. In JSON, a coordinate written to fifteen significant figures is roughly twice the characters of one written to six — and measured on a comparable payload, rounding removed 45%.

layer_data = layer_data.assign(lon=layer_data.lon.round(6),
                               lat=layer_data.lat.round(6))

4. Aggregate before sending, above about a hundred thousand #

A million overlapping points renders as a solid shape. Aggregating produces a map that reads better and a payload three orders of magnitude smaller:

13,464,017 points → 961,781 cells at 0.1°
                  →  71,921 cells at 0.5°
                  →  23,422 cells at 1.0°

Use GridLayer on aggregated cells rather than HexagonLayer on raw points when the data is large: the first sends thousands of rows, the second sends millions and bins them in the browser.

5. Use binary attributes when the points must stay individual #

deck.gl accepts typed arrays, which skips JSON parsing entirely. Measured on a million coordinate pairs:

JSON              31.77 MB
JSON, gzipped      6.86 MB
Arrow + zstd      11.53 MB
raw float32        8.00 MB

In a Streamlit app the practical route is to serve the binary from an endpoint and point the layer at it, rather than embedding it in the page.

6. Keep the view state, and set pixel bounds #

if "view" not in st.session_state:
    st.session_state.view = pdk.ViewState(latitude=54, longitude=-2, zoom=5)

layer = pdk.Layer("ScatterplotLayer", data=..., get_radius=30,
                  radius_min_pixels=1, radius_max_pixels=8)

get_radius is in metres, so at low zoom a 30 m point is sub-pixel and invisible. radius_min_pixels keeps it drawn; radius_max_pixels stops it becoming a disc at high zoom.

Code examples #

Example 1 — a scatter layer that scales #

import pydeck as pdk
import streamlit as st


def scatter_deck(points, view, colour_column=None, radius_m=30):
    """Only the columns the layer reads, rounded, with pixel bounds set."""
    columns = ["lon", "lat"] + ([colour_column] if colour_column else [])
    data = points[columns].assign(
        lon=points["lon"].round(6), lat=points["lat"].round(6))

    layer = pdk.Layer(
        "ScatterplotLayer",
        data=data,
        get_position=["lon", "lat"],
        get_radius=radius_m,
        radius_min_pixels=1,
        radius_max_pixels=10,
        get_fill_color=(f"[255 * {colour_column}, 120, 200 - 255 * {colour_column}, 160]"
                        if colour_column else [14, 165, 233, 140]),
        pickable=True,
    )
    return pdk.Deck(layers=[layer], initial_view_state=view,
                    map_style="light",
                    tooltip={"text": "{lon}, {lat}"})

Example 2 — aggregating server-side and drawing cells #

import numpy as np
import pandas as pd
import pydeck as pdk


def grid_deck(points, view, cell_deg=0.05, value=None):
    """Bin in Python, draw a few thousand cells — not a million points."""
    binned = points.assign(
        lon=(np.floor(points.lon / cell_deg) * cell_deg).round(6),
        lat=(np.floor(points.lat / cell_deg) * cell_deg).round(6))

    aggregations = {"n": ("lon", "size")}
    if value:
        aggregations["mean"] = (value, "mean")
    cells = binned.groupby(["lon", "lat"], as_index=False).agg(**aggregations)

    print(f"{len(points):,} points → {len(cells):,} cells "
          f"({100 * len(cells) / max(1, len(points)):.2f}%)")

    top = cells["n"].quantile(0.98)
    layer = pdk.Layer(
        "ColumnLayer", data=cells,
        get_position=["lon", "lat"],
        get_elevation="n", elevation_scale=50,
        radius=int(cell_deg * 111_000 / 2),
        get_fill_color=f"[255 * n / {top}, 140, 220 - 200 * n / {top}, 200]",
        pickable=True, auto_highlight=True)

    return pdk.Deck(layers=[layer], initial_view_state=view,
                    tooltip={"text": "{n} points"})

Example 3 — measuring the page before shipping it #

import gzip


def deck_page_size(deck, label=""):
    """pydeck embeds the data in the HTML — measure it."""
    html = deck.to_html(as_string=True, notebook_display=False).encode()
    raw, gz = len(html) / 1e6, len(gzip.compress(html, 6)) / 1e6
    verdict = ("fine" if raw < 5 else
               "heavy" if raw < 20 else
               "will not load reliably")
    print(f"{label:24} {raw:8.2f} MB  ({gz:.2f} MB gzipped)  {verdict}")
    return raw, gz

1,000 points               0.08 MB  (0.01 MB gzipped)  fine
10,000 points              0.74 MB  (0.09 MB gzipped)  fine
100,000 points             7.39 MB  (0.84 MB gzipped)  heavy
1,000,000 points          73.78 MB  (7.95 MB gzipped)  will not load reliably

Running this against the actual layer, at the actual size, is a minute of work that prevents the most common late failure in a map app.

Explanation #

Why WebGL changes the ceiling and not the transport #

deck.gl draws points as GPU primitives, so drawing a million is a graphics problem the hardware is designed for. The DOM-based approach builds an element per feature, which is why folium's measured cost is about 478 bytes and a fraction of a millisecond of layout per marker.

But the data still has to reach the browser. pydeck's page embeds it as JSON, so the transport cost is unchanged by the renderer — 73.78 MB for a million points. That is why the fixes at scale are about encoding and aggregation rather than about the layer.

Why aggregation improves the map as well as the payload #

At a million points on a country-sized map, every pixel contains many points and the display is a solid shape. The information the reader can extract is "there are a lot of points", which a much smaller aggregation conveys better.

The measured ladder makes the trade concrete: 13.5 million points binned to 23,422 one-degree cells is 0.17% of the rows and a map that shows the pattern. The aggregation is not a compromise forced by the browser; it is the better cartography that happens to be cheap.

Why radius_min_pixels matters so much #

get_radius in ScatterplotLayer is in metres by default, which is correct and surprising: a 30 m radius at zoom 4 is far smaller than a pixel, so the layer renders and nothing appears.

The usual bug report is "pydeck shows nothing", and the usual cause is this. Setting radius_min_pixels=1 guarantees a visible mark at every zoom, and radius_max_pixels stops each point becoming a disc when zoomed in.

Why binary attributes are the real answer above a million #

Even gzipped, a million points as JSON is 6.86 MB on the wire and — more importantly — several seconds of parsing on the browser's main thread.

Typed arrays remove the parsing entirely: the same coordinates as float32 are 8.00 MB uncompressed and go straight into a GPU buffer. In an app that means serving the array from an endpoint rather than embedding it, which is a different architecture and the one that scales past a page.

Edge cases or notes #

• get_radius is in metres. Set radius_min_pixels or low zooms render nothing.
• pydeck embeds the data in the HTML — the page size is the payload.
• Send only the columns the layer reads; every column travels.
• GridLayer and HexagonLayer aggregate in the browser, so the raw points are still sent.
• ScreenGridLayer is the cheapest for very large sets: it bins at screen resolution.
• pickable=True costs memory — it keeps the source rows for tooltips.
• Colour expressions are JavaScript strings evaluated per feature; keep them simple.
• Measure to_html() at the real size before shipping.

Internal links #

• What a map app can afford to send to the browser — the budget these numbers set
• How to build a Streamlit app with an interactive map — where the deck goes
• Fixing a map app that takes ten seconds to load — the symptom of ignoring the page size
• How to make a hexbin map of points — aggregation done in Python
• How to aggregate millions of points into a grid with DuckDB — aggregating at source
• How to prepare a GeoDataFrame for the web — the reductions
• Point density explained — what an aggregated map means
• How to make interactive maps with folium — the component pydeck replaces

FAQ #

How many points can pydeck handle? #

Rendering, hundreds of thousands comfortably. The limit is transport: the page embeds the data, so 100,000 points is a 7.39 MB page and a million is 73.78 MB.

Why does my pydeck layer render nothing? #

Almost always get_radius in metres at a low zoom, where the points are sub-pixel. Set radius_min_pixels=1.

When should I aggregate instead? #

Above roughly a hundred thousand points. It reduces the payload by orders of magnitude and produces a more readable map — a million overlapping points is a solid blob.

Is pydeck better than folium? #

For scale, decisively: folium builds a DOM object per feature, measured at about 478 bytes each, so 50,000 markers is 23.88 MB. For Leaflet plugins such as drawing tools, folium is still the right choice.

How do I get past the page-size limit? #

Serve the data from an endpoint as a typed array rather than embedding it. A million coordinate pairs is 31.77 MB as JSON and 8.00 MB as float32, with no parsing.

Do the aggregating layers help with the payload? #

No — GridLayer and HexagonLayer bin in the browser, so the raw points are still sent. Aggregate in Python if the payload is the problem.