🧭 Concept Gridded & climate data Files & formats Gridded & Climate Data · step 1 of 20 Updated

NetCDF and Gridded Data Explained: Dimensions, Variables and Attributes

Problem statement

A NetCDF file looks like a black box until something goes wrong: a variable whose values are a hundred times too large, a time axis in "hours since 1800", a read that takes a hundred times longer than it should, or a library that refuses to open the file at all. Every one of those comes from a part of the format that is easy to overlook — how the arrays are laid out, what the attributes say, and how the numbers are stored on disk.

Measured on three public files, NOAA's NCEP/NCAR Reanalysis 1 monthly air temperature, its 6-hourly 2024 file, and one day of NOAA OISST v2.1 sea surface temperature:

  • Opening is not reading. The 29.6 MB monthly file opened in 5.3 ms; loading its 938 × 73 × 144 array took 42.1 ms and 39.4 MB of memory.
  • Stored values are not the values. OISST keeps temperature as 16-bit integers: the stored integers averaged 609.561; the decoded temperatures averaged 14.140 °C.
  • Files are often compressed. The 6-hourly 2024 file was 22.2 MB on disk and 61.6 MB once read.
  • The chunk layout decides what is fast. Written with one chunk per month, one month's map read in 0.95 ms and one cell's 938-month series in 96.74 ms. Written with chunks running through time, the series took 1.59 ms and the map 65.32 ms.

Quick answer

A NetCDF file holds named dimensions, variables that span some of those dimensions, and attributes that describe both. Variables named after a dimension are its coordinates. Open the file and print it to see all four:

import xarray as xr

ds = xr.open_dataset("air.mon.mean.nc")
print(ds)
print(ds["air"].encoding["dtype"], ds["air"].encoding["chunksizes"])
<xarray.Dataset> Size: 39MB
Dimensions:  (time: 938, lat: 73, lon: 144)
Coordinates:
  * time     (time) datetime64[ns] 8kB 1948-01-01 1948-02-01 ... 2026-02-01
  * lat      (lat) float32 292B 90.0 87.5 85.0 82.5 ... -82.5 -85.0 -87.5 -90.0
  * lon      (lon) float32 576B 0.0 2.5 5.0 7.5 10.0 ... 350.0 352.5 355.0 357.5
Data variables:
    air      (time, lat, lon) float32 39MB ...
Attributes:
    description:    Data from NCEP initialized reanalysis (4x/day).  These ar...
    platform:       Model
    Conventions:    COARDS
    NCO:            20121012
    history:        Thu May  4 20:11:16 2000: ncrcat -d time,0,623 /Datasets/...
    title:          monthly mean air.sig995 from the NCEP Reanalysis
    dataset_title:  NCEP-NCAR Reanalysis 1
    References:     http://www.psl.noaa.gov/data/gridded/data.ncep.reanalysis...
float32 (1, 73, 144)

The fifth thing, the encoding, is not printed: it records how the values sit on disk — data type, packing, fill value, compression and chunks — and xarray undoes it as it reads.

Table of the parts of a NetCDF file — dimensions, coordinate variables, data variables, attributes and encoding — with what each held in the NCEP monthly air temperature file.
Four parts are printed when you open a file; the fifth, the encoding, decides what the numbers on disk mean.

Step-by-step solution

1. Dimensions name the axes

A dimension is a name and a length: time 938, lat 73 and lon 144 in the monthly file. Dimensions carry no values of their own. One dimension per file may be unlimited, meaning new records can be appended along it; here that is time, which is how a monthly file can grow from 1948 to 2026 without being rewritten.

2. Coordinate variables label the axes

A variable with the same name as a dimension is that dimension's coordinate. lat holds 73 values from 90.0 down to −90.0, lon 144 values from 0.0 to 357.5, and time 938 monthly timestamps. Their order and range matter more than any other detail in the file, because every selection is made against them — see selecting a time range and location.

3. Data variables share those dimensions

air spans (time, lat, lon) and is stored as 32-bit floats: 938 × 73 × 144 × 4 bytes is 39.4 MB. A file can hold several data variables on the same dimensions — the OISST file has four, sst, anom, err and ice, each (time, zlev, lat, lon) with lengths 1, 1, 720 and 1440.

4. Attributes say what the numbers mean

Attributes are small name–value pairs on a variable or on the whole file. air has units = "degC" and a long_name; the file declares Conventions = "COARDS", the ancestor of the CF conventions that most climate files follow now. Software relies on attributes to decode times, mask missing values and recognise latitude and longitude, so a missing or wrong attribute causes wrong results rather than an error — the subject of CF conventions explained.

5. Stored values are encoded

The time variable stores 1297320.0 for its first value, with the unit "hours since 1800-01-01"; decoded, that is 1 January 1948. OISST stores temperature as 16-bit integers with scale_factor = 0.01 and _FillValue = -999: one tropical Pacific cell is stored as 3083 and means 30.83 °C, and land cells hold −999 — 345,650 of the 1,036,800 cells, 33.3%. xarray applies all of this on read. Code that reads the raw integers, or decodes them by hand and forgets the fill value, gets 609.561 or 6.096 instead of 14.140.

6. The format version decides who can read it

"NetCDF" covers two storage formats. The classic format is a simple binary layout; NetCDF-4 stores the same model inside HDF5, which adds compression, chunking and groups. The monthly file is NetCDF-4 (using the classic data model inside HDF5), and SciPy's reader, which understands only the classic format, refused it with is not a valid NetCDF 3 file. The h5netcdf engine can read NetCDF-4 but needs the h5py package installed.

7. Compression and chunks decide size and speed

NetCDF-4 variables are split into chunks, and each chunk can be compressed. The 6-hourly 2024 file was 22.2 MB on disk and 61.6 MB decoded; the OISST day was 1.5 MB on disk and 16.6 MB decoded. A read decompresses every chunk it touches, so the chunk shape should match the way the data will be read (Example 3).

8. Open lazily, then load what you need

xr.open_dataset reads the header and the coordinates, not the data. Reading the header with netCDF4 took 3.1 ms, opening with xarray 5.3 ms, and loading everything 42.1 ms. Select first and load second, and only the chunks you selected are read — see lazy loading explained.

Flow from a stored 16-bit integer through the fill-value check, scale factor and offset to a decoded sea surface temperature.
xarray applies these steps on every read; skipping any one of them gives a plausible-looking wrong number.

Code examples

Example 1 — list the structure without reading any data 

import netCDF4

with netCDF4.Dataset("air.mon.mean.nc") as nc:
    print(nc.data_model, "groups:", list(nc.groups))
    for name, dim in nc.dimensions.items():
        print(f"dimension {name:4} length {len(dim):4}  unlimited {dim.isunlimited()}")
    for name, var in nc.variables.items():
        print(f"variable  {name:4} {str(var.dtype):8} {var.dimensions}  chunks {var.chunking()}  "
              f"units {getattr(var, 'units', '-')}")
    print("global attributes:", nc.ncattrs())
NETCDF4_CLASSIC groups: []
dimension lat  length   73  unlimited False
dimension lon  length  144  unlimited False
dimension time length  938  unlimited True
variable  lat  float32  ('lat',)  chunks contiguous  units degrees_north
variable  lon  float32  ('lon',)  chunks contiguous  units degrees_east
variable  time float64  ('time',)  chunks [1]  units hours since 1800-01-01 00:00:0.0
variable  air  float32  ('time', 'lat', 'lon')  chunks [1, 73, 144]  units degC
global attributes: ['description', 'platform', 'Conventions', 'NCO', 'history', 'title', 'dataset_title', 'References']

The netCDF4 library reads only the header here, so this is safe on a file of any size. The data variable is stored one month per chunk.

Example 2 — stored integers against decoded values 

import xarray as xr

path = "oisst-avhrr-v02r01.20240115.nc"
raw = xr.open_dataset(path, mask_and_scale=False)["sst"]
decoded = xr.open_dataset(path)["sst"]

fill, scale = raw.attrs["_FillValue"], raw.attrs["scale_factor"]
cell = dict(lat=0.125, lon=180.125)
print(f"stored {raw.dtype}, decoded {decoded.dtype}; scale_factor {scale:.2f}, _FillValue {fill}")
print(f"one cell: stored {int(raw.sel(cell).squeeze())}, decoded {float(decoded.sel(cell).squeeze()):.2f}")
print(f"fill cells: {int((raw == fill).sum()):,} of {raw.size:,}")
print(f"mean of stored integers   {float(raw.mean()):9.3f}")
print(f"stored x scale_factor     {float(raw.mean()) * scale:9.3f}")
print(f"mean of decoded values    {float(decoded.mean()):9.3f}")
stored int16, decoded float32; scale_factor 0.01, _FillValue -999
one cell: stored 3083, decoded 30.83
fill cells: 345,650 of 1,036,800
mean of stored integers     609.561
stored x scale_factor         6.096
mean of decoded values       14.140

With mask_and_scale=False, the packing attributes stay in attrs; decoded, they move to encoding. The decoded mean here is a plain mean over cells, not an area-weighted one.

Example 3 — the same data, three chunk layouts 

import time

import netCDF4
import xarray as xr


def fastest_read(path, read, repeats=5):
    timings = []
    for _ in range(repeats):
        start = time.perf_counter()
        with netCDF4.Dataset(path) as nc:
            read(nc["air"])
        timings.append(time.perf_counter() - start)
    return min(timings) * 1000


ds = xr.open_dataset("air.mon.mean.nc")
for chunks in [(1, 73, 144), (120, 73, 144), (938, 8, 8)]:
    path = "air_chunks_{}_{}_{}.nc".format(*chunks)
    ds.to_netcdf(path, encoding={"air": {"zlib": True, "complevel": 2, "chunksizes": chunks}})
    one_map = fastest_read(path, lambda v: v[500, :, :])
    one_series = fastest_read(path, lambda v: v[:, 20, 100])
    print(f"chunks {str(chunks):15} one month's map {one_map:6.2f} ms   one cell's 938 months {one_series:6.2f} ms")
chunks (1, 73, 144)    one month's map   0.95 ms   one cell's 938 months  96.74 ms
chunks (120, 73, 144)  one month's map  14.84 ms   one cell's 938 months 102.56 ms
chunks (938, 8, 8)     one month's map  65.32 ms   one cell's 938 months   1.59 ms

The three files were 29.6, 28.7 and 28.7 MB and read back identical to the original. Passing encoding for air replaces the encoding the variable was read with, so the rewritten files contain exactly the original values in the new layout.

Explanation

Why the format describes itself

A NetCDF file carries the names, shapes, types and meanings of its arrays inside it, so it can be read without a separate description. That is why climate and weather data settled on it: a 1948–2026 reanalysis has to stay readable by software that did not exist when it was written. The cost is that the description can be incomplete or wrong, and the file will still open.

Why coordinates are separate variables

Keeping labels in their own variables lets many data variables share one set of coordinates, and lets the coordinates have their own attributes — units, axis, standard_name, bounds. It also means nothing forces them to be sorted, evenly spaced or in a particular convention. Descending latitude and 0–360 longitude are both legal and both common.

Why values are packed

A 16-bit integer takes half the space of a 32-bit float. OISST's four variables would be 16.6 MB as floats; as packed integers they are half that before compression, and a scale factor of 0.01 keeps a precision of 0.01 °C, finer than the measurement itself. The price is that every reader has to apply the scale factor and mask the fill value, and some do not.

Why chunk shape matters so much

A chunk is the unit of storage and of decompression. Reading one cell's time series from a file chunked one month at a time touches all 938 chunks and decompresses 938 maps to extract 938 numbers. Reading one map from a file chunked as 8 × 8 columns through time touches every column chunk. Neither layout is wrong; each is right for one access pattern.

Bar chart of the time to read one month's map and one cell's 938-month series from the same data written with three chunk layouts.
The data were identical; only the chunk shape changed which read was cheap.

Edge cases or notes

  • Groups organise variables hierarchically in NetCDF-4; xr.open_dataset reads the root group unless you pass group=.
  • Two-dimensional coordinates are normal for curvilinear model grids; lat and lon are then data-like variables with their own dimensions.
  • Bounds variables such as time_bnds describe cell edges and are easy to drop by accident.
  • Remote files are slow to open cell by cell. Opening the 29.6 MB monthly file over HTTP byte ranges took 185.8 seconds; downloading it whole took 6.06.
  • An .nc extension does not guarantee a NetCDF file. A failed download saved as .nc can be an HTML error page.
  • Re-saving can change values. An encoding inherited from the source file travels with the data unless you replace it.
  • GRIB and Zarr hold the same kind of data with different trade-offs; see GRIB, NetCDF or Zarr.

FAQ

What is a NetCDF file?

A self-describing file of named multi-dimensional arrays. It holds dimensions, variables that span them, coordinate variables that label them, attributes that describe everything, and an encoding that records how values are stored.

What is the difference between a dimension and a coordinate?

A dimension is a name and a length, such as lat with 73 steps. A coordinate is a variable with the same name that holds the 73 latitude values.

Why are the raw values in my NetCDF file integers?

They are packed to save space. OISST stores temperature as 16-bit integers with a scale factor of 0.01, so a stored 3083 means 30.83 °C, and −999 marks land.

What is the difference between NetCDF-3 and NetCDF-4?

NetCDF-3 is the classic binary format; NetCDF-4 stores the same data model in HDF5 and adds compression, chunking and groups. SciPy's reader handles only NetCDF-3.

Why is reading one point's time series so slow?

Because the file is chunked for reading maps. With one chunk per month, a single cell's 938-month series has to decompress all 938 chunks.

Does opening a NetCDF file with xarray load the data?

No. It reads the header and coordinates; the 29.6 MB monthly file opened in 5.3 ms and loaded in 42.1 ms. Data are read when you compute or load.