Cloud-Optimised GeoTIFF Explained: Tiles, Overviews and Range Requests
Problem statement
A Cloud-Optimised GeoTIFF is an ordinary GeoTIFF arranged so a client can read part of it over HTTP without downloading it. Three properties do that, and a file missing any one behaves like an ordinary file.
Measured on the same 4096 × 4096 uint16 raster written three ways and served over HTTP:
one 512 x 512 window at full resolution
COG (512 tiles + overviews) 5 requests 1.585 MB
tiled, no overviews 3 requests 1.557 MB
striped 2 requests 3.244 MB
the whole scene at 1/16 resolution
COG 1 request 0.115 MB
tiled, no overviews 3 requests 24.672 MB
Tiling halves the bytes for a window. Overviews cut a thumbnail by a factor of 215.
Quick answer
import rasterio
from rasterio.enums import Resampling
def write_cog(src_path, dst_path, blocksize=512, levels=(2, 4, 8, 16)):
with rasterio.open(src_path) as src:
profile = src.profile | {
"driver": "GTiff", "tiled": True,
"blockxsize": blocksize, "blockysize": blocksize,
"compress": "deflate", "predictor": 2,
"BIGTIFF": "IF_SAFER",
}
with rasterio.open(dst_path, "w", **profile) as dst:
for band in range(1, src.count + 1):
dst.write(src.read(band), band)
dst.build_overviews(list(levels), Resampling.average)
dst.update_tags(ns="rio_overview", resampling="average")Check it afterwards:
with rasterio.open(dst_path) as ds:
print(ds.block_shapes[0], ds.overviews(1))(512, 512) [2, 4, 8, 16]
An empty overview list means the file is tiled and not cloud-optimised.
Step-by-step solution
1. Tile the image
A striped GeoTIFF stores whole rows. A 512-pixel-wide window on a 4096-wide image touches 512 strips, each 4096 pixels long — so 87% of every strip is discarded.
The measurement shows exactly that: the striped file transferred 3.244 MB against the tiled file's 1.557 MB for identical output pixels.
512 × 512 is a good default. Smaller tiles mean more requests; larger ones mean more waste on small windows.
2. Build overviews
Overviews are pre-computed downsampled copies stored in the same file. Without them, a client asked for a thumbnail must read every full-resolution pixel and average them itself.
whole scene at 1/16
with overviews 1 request 0.115 MB
without 3 requests 24.672 MB
Use factors of 2 down to a level that fits in a single tile. For a 4096-pixel image, [2, 4, 8, 16] takes the smallest level to 256 pixels — one tile.
Choose the resampling method for the data: average for continuous values, nearest for classes, mode for categorical rasters where averaging is meaningless.
3. Put the header at the front
A client reads the header first, so it should be one contiguous block at the start of the file, and it should include the tile offset table.
GDAL's dedicated COG driver guarantees this layout. Writing with GTiff and adding overviews afterwards usually produces an acceptable file — usually, because the overview data may land after the main image, which costs one extra seek rather than correctness.
4. Compress, and pick a predictor
Deflate with predictor=2 (horizontal differencing) is a strong default for continuous integer data. Measured on the real Sentinel-2 band used here, the COG was 33.4 MB against 34.4 MB raw for a 4096 × 4096 uint16 crop — modest, because satellite imagery is noisy and compresses poorly.
For elevation, classifications and anything smooth, compression ratios are far better. For floating-point data, predictor=3 is the floating-point variant.
5. Validate rather than assume
def is_cog(path):
with rasterio.open(path) as ds:
tiled = ds.block_shapes[0][0] not in (1, ds.height)
overviews = bool(ds.overviews(1))
print(f" tiled: {tiled} ({ds.block_shapes[0]})")
print(f" overviews: {ds.overviews(1) or 'NONE'}")
return tiled and overviews
Code examples
Example 1 — a writer that validates its own output
import os
import rasterio
from rasterio.enums import Resampling
RESAMPLING = {"continuous": Resampling.average,
"categorical": Resampling.mode,
"classified": Resampling.nearest}
def write_cog(src_path, dst_path, kind="continuous", blocksize=512,
compress="deflate", predictor=None, min_overview_px=256):
"""Write a COG, choosing overview levels and resampling from the data."""
with rasterio.open(src_path) as src:
largest = max(src.width, src.height)
levels, factor = [], 2
while largest / factor >= min_overview_px:
levels.append(factor)
factor *= 2
if predictor is None:
predictor = 3 if src.dtypes[0].startswith("float") else 2
profile = src.profile | {
"driver": "GTiff", "tiled": True,
"blockxsize": blocksize, "blockysize": blocksize,
"compress": compress, "predictor": predictor,
"BIGTIFF": "IF_SAFER",
}
with rasterio.open(dst_path, "w", **profile) as dst:
for band in range(1, src.count + 1):
dst.write(src.read(band), band)
if levels:
dst.build_overviews(levels, RESAMPLING[kind])
dst.update_tags(ns="rio_overview", resampling=kind)
with rasterio.open(dst_path) as check:
ok = bool(check.overviews(1)) and check.block_shapes[0][0] != 1
print(f" {os.path.getsize(dst_path) / 1e6:7.2f} MB, "
f"blocks {check.block_shapes[0]}, overviews {check.overviews(1)}")
if not ok:
raise RuntimeError("output is not cloud-optimised")
return dst_pathDeriving the overview levels from the image size means the smallest level always fits in one tile, which is what makes a full-extent thumbnail a single request.
Example 2 — measuring what a window costs
import time
import rasterio
from rasterio.windows import Window
def window_cost(url, sizes=(1, 256, 512, 1024, 2048), offset=1792):
"""Bytes and requests scale with tiles touched, not pixels asked for."""
for size in sizes:
start = time.time()
with rasterio.Env(GDAL_DISABLE_READDIR_ON_OPEN="EMPTY_DIR",
CPL_VSIL_CURL_ALLOWED_EXTENSIONS=".tif"):
with rasterio.open(url) as ds:
ds.read(1, window=Window(offset, offset, size, size))
print(f" {size:5d}x{size:<5d} {(time.time() - start) * 1000:7.0f} ms")Measured against a local byte-counting server, on a COG with 512-pixel tiles:
1 px 4 requests 470.8 kB 470,793 bytes/px
64 px 4 requests 470.8 kB 115 bytes/px
256 px 4 requests 470.8 kB 7.2 bytes/px
512 px 5 requests 1,585.3 kB 6.0 bytes/px
1024 px 6 requests 3,447.0 kB 3.3 bytes/px
2048 px 8 requests 9,619.6 kB 2.3 bytes/px
A single pixel costs 470 kB, and so does a 256 × 256 window — both read exactly one tile plus the header. The tile is the unit of transfer. Reading pixels one at a time is the worst possible access pattern; reading in tile-aligned blocks is the best.
Example 3 — checking a remote file before relying on it
import rasterio
def inspect_remote(url):
"""What will reads from this file cost?"""
with rasterio.Env(GDAL_DISABLE_READDIR_ON_OPEN="EMPTY_DIR",
CPL_VSIL_CURL_ALLOWED_EXTENSIONS=".tif"):
with rasterio.open(url) as ds:
block = ds.block_shapes[0]
overviews = ds.overviews(1)
print(f" {ds.width} x {ds.height} {ds.dtypes[0]}, {ds.count} band(s)")
print(f" blocks {block}, compression "
f"{ds.profile.get('compress')}, overviews {overviews or 'NONE'}")
problems = []
if block[0] == 1 or block[0] == ds.height:
problems.append("striped: windowed reads will fetch whole rows")
if not overviews:
problems.append("no overviews: downsampled reads cost full "
"resolution")
smallest = max(ds.width, ds.height) / max(overviews or [1])
if overviews and smallest > 2 * max(block):
problems.append(f"coarsest overview is still {smallest:.0f} px — "
"a full-extent view needs several requests")
for p in problems:
print(f" ! {p}")
return not problemsRun this once on a data source before building a pipeline against it. A striped remote GeoTIFF is not unusable, but it changes the arithmetic of every design decision downstream.
Explanation
Why the tile is the unit of transfer
A COG's internal tiles are compressed independently, so a client cannot read part of one — it must fetch the whole compressed block and decompress it.
That is why a 1 × 1 pixel read and a 256 × 256 read both cost 470 kB: both touch exactly one 512 × 512 tile. The per-pixel cost falls from 470,793 bytes to 7.2 bytes across that range, purely from amortising one tile over more pixels.
The design consequence is that access patterns should be tile-shaped. Sampling a raster at a thousand scattered points is a thousand tile reads; sampling it over a block is one.
Why overviews matter more than tiling
Tiling saved a factor of two on the window read. Overviews saved a factor of 215 on the thumbnail.
The asymmetry comes from what each avoids. Tiling avoids reading pixels beside the ones you want, which is bounded by the aspect ratio of a strip. Overviews avoid reading pixels you will immediately average away, which is bounded by the downsampling factor squared — 256 at 1/16.
Any zoomable application spends most of its reads zoomed out, so this dominates.
Why the header layout matters
A client opening a remote file makes a speculative first read — GDAL asks for the first 16 kB by default. If the header, the tile offsets and the overview headers all fit in that, the file is open after one request.
If they are scattered, the client makes several round trips before reading any data. Against an object store at 100 ms latency, four extra round trips is nearly half a second on every file opened.
Measured, opening a well-formed COG and reading nothing cost 2 requests and 28 kB.
Why compression choice is not obvious
Deflate with a predictor is the safe default: universally supported, decent ratios, moderate CPU.
The alternatives trade differently. LZW is faster to decompress and compresses less. ZSTD is faster and better but needs a recent GDAL. JPEG and WEBP are lossy and unacceptable for measurements, though fine for visual products. LERC is lossy with a controllable error bound, which suits elevation.
The measured 33.4 MB against 34.4 MB raw for Sentinel-2 reflectance is a reminder that noisy 16-bit imagery barely compresses. Elevation and classifications compress far better, and the same settings can give a fivefold difference on different data.
Edge cases or notes
- Tiled without overviews is not a COG. Validate both.
- 512 × 512 tiles are a good default; 256 for small images.
- Overview levels should reach one tile. Otherwise a full-extent view is several requests.
averagefor continuous,modeornearestfor categorical overviews.predictor=2for integers,3for floats. Wrong predictor makes files larger.- A 1-pixel read costs a whole tile — 470 kB here.
BIGTIFF=IF_SAFERavoids the 4 GB limit surprising you on a large write.- Use GDAL's
COGdriver where available; it guarantees the layout.
Internal links
- Cloud-native geospatial explained — the pattern across formats
- How to write a cloud-optimised GeoTIFF in Python — the full writer
- How to read a COG from a URL without downloading the whole file — the reading side
- Reading a COG from a URL is slow or downloads everything — diagnosing bad performance
- How to choose chunk and tile sizes that actually help — sizing the blocks
- Raster resampling explained — what overview resampling does
- The raster data model explained — what the header holds
- How to reduce GIS file size in Python without wrecking the data — compression trade-offs
FAQ
What makes a GeoTIFF cloud-optimised?
Internal tiling, pre-computed overviews, and a header laid out so a client can read it in one request. All three, not one.
How much do overviews save?
Measured, a whole-scene thumbnail cost 0.115 MB with overviews and 24.672 MB without — the entire file — a factor of 215.
What tile size should I use?
512 × 512 for most imagery, 256 for small rasters. Smaller means more requests; larger means more waste on small windows.
Why does reading one pixel cost so much?
Because tiles are compressed independently, so the smallest readable unit is a whole tile — 470 kB on a 512-tile COG. Read in tile-aligned blocks instead.
Which compression should I use?
Deflate with predictor=2 for integers and 3 for floats is a safe default. ZSTD is better if your readers support it; never use JPEG for measurements.
Do I need GDAL's COG driver?
It guarantees the layout. Writing with GTiff plus build_overviews usually produces an acceptable file — validate the result either way.
How do I check whether a file is a COG?
Open it and check ds.block_shapes for tiling and ds.overviews(1) for overviews. An empty overview list is the common failure.