🧭 Concept Gridded & climate data Files & formats Gridded & Climate Data · step 2 of 20 Updated

CF Conventions Explained: How a NetCDF File Says What Its Numbers Mean

Problem statement

A NetCDF file can hold any array with any attributes. Whether software understands it — decodes the times, hides the missing values, finds the latitude axis, knows a value is a monthly mean rather than an instant — depends on attributes that follow the Climate and Forecast (CF) metadata conventions. When those attributes are missing, misspelled or misread, the file still opens. The mistakes appear later, as wrong numbers.

Three widely used files, checked attribute by attribute:

  • NCEP/NCAR Reanalysis 1 monthly air temperature declares the older COARDS conventions: units and a missing_value, but no standard_name, no calendar attribute and no cell bounds.
  • NOAA OISST v2.1 declares CF-1.6: packed 16-bit values with a scale factor and fill value, but its latitude and longitude carry only long_name and units — and rioxarray could not identify them as spatial axes until a standard_name or axis attribute was added.
  • A CMIP6 GFDL-ESM4 temperature file declares CF-1.7 and carries almost everything: standard_name, cell_methods = "area: time: mean", bounds for latitude and time, and a noleap calendar in which 29 February does not exist.

And one attribute that looks authoritative does nothing on its own: xarray does not apply valid_min or valid_max. A test file with a valid range of 0 to 100 read back −5 and 999 unchanged.

Quick answer

CF is a set of attribute names and meanings. Before trusting a file, read its attributes with decoding switched off and check the ones your analysis depends on:

import xarray as xr

raw = xr.open_dataset("oisst-avhrr-v02r01.20240115.nc", decode_cf=False)
print(raw.attrs.get("Conventions"))
print({k: raw["sst"].attrs.get(k) for k in ("standard_name", "units", "_FillValue", "scale_factor", "valid_min")})
print({k: raw["lat"].attrs.get(k) for k in ("standard_name", "units", "axis")})
print({k: raw["time"].attrs.get(k) for k in ("units", "calendar", "bounds")})
CF-1.6, ACDD-1.3
{'standard_name': None, 'units': 'Celsius', '_FillValue': np.int16(-999), 'scale_factor': np.float32(0.01), 'valid_min': np.int16(-300)}
{'standard_name': None, 'units': 'degrees_north', 'axis': None}
{'units': 'days since 1978-01-01 12:00:00', 'calendar': None, 'bounds': None}

The attributes that change numbers are _FillValue and missing_value, scale_factor and add_offset, time units and calendar, and valid_range; the ones that change what tools recognise are standard_name, units and axis on the coordinates; the ones that change interpretation are cell_methods and bounds.

Table of which CF attributes three real files carry: NCEP Reanalysis, NOAA OISST and a CMIP6 model output.
Each file "follows the conventions", and each leaves out something a tool downstream needs.

Step-by-step solution

1. Read the Conventions attribute, then check anyway

The global Conventions attribute declares what a file intends to follow: COARDS for the NCEP file, CF-1.6, ACDD-1.3 for OISST, CF-1.7 CMIP-6.0 UGRID-1.0 for the CMIP6 file. It is a promise, not a validation. Every file above was missing at least one attribute that CF recommends for its kind of data.

2. Identify the quantity with units and standard_name

units must be a string that the UDUNITS library understands — degC, Celsius and K all are. standard_name names the physical quantity from CF's controlled list, so air_temperature means the same thing in every file. Only the CMIP6 file had one. Without it, software has to guess from variable names, and a variable called air does not say what it holds.

3. Mark missing data with _FillValue or missing_value

Both attributes name a value that means "no data". OISST uses _FillValue = -999; the NCEP file uses missing_value = -9.96921e+36. xarray treats both the same way and turns them into NaN: a test value equal to missing_value came back as NaN. In the NCEP file no cell actually held the missing value, so the attribute was a declaration with nothing to mask.

4. Unpack with scale_factor and add_offset — and read the valid range in packed units

OISST's scale_factor = 0.01 turns stored integers into degrees. Its valid_min and valid_max are −300 and 4500: limits on the stored integers, meaning −3.00 °C to 45.00 °C. After decoding, xarray keeps those two attributes on the variable unchanged, next to values in degrees, and never applies them. If the valid range matters, apply it yourself and convert the limits first (Example 3).

5. Decode time with units and calendar

Time is a number and a reference: hours since 1800-01-01 00:00:0.0 for NCEP, days since 1978-01-01 12:00:00 for OISST. The calendar attribute says how to count days. NCEP and OISST omit it, which CF defines as the standard Gregorian calendar. The CMIP6 file uses noleap, in which every year has 365 days: 165 model years hold 60,225 days, 40 fewer than 1850–2014 in the real calendar, and cftime refuses to create 29 February 2000 at all.

6. Label the axes so tools can find them

Libraries look for latitude and longitude through attributes: standard_name of latitude and longitude, units of degrees_north and degrees_east, or axis of Y and X. The NCEP coordinates carry standard_name and axis, and rioxarray found them. OISST's carry only long_name and units, and rioxarray raised MissingSpatialDimensionError; adding either standard_name or axis was enough (Example 2). Renaming the dimensions to latitude/longitude or y/x also worked.

7. Read cell_methods and bounds before combining values

cell_methods = "area: time: mean" says each CMIP6 value is an average over the grid cell and over the time step, not a sample at the centre. bounds = "lat_bnds" and "time_bnds" point to variables holding cell edges, which is what an exact area weight or a correct monthly resample needs. NCEP and OISST have neither, so their cell edges have to be inferred from the spacing.

8. Audit, then fix or document

Run a short audit on every new source (Example 1). Add attributes that tools need, record ones you infer, and never assume that a declared convention means a complete one.

Table of CF attributes and whether xarray applies each automatically when opening a file.
xarray decodes what changes the numbers on disk into values; it leaves meaning — units, valid ranges, cell methods — to you.

Code examples

Example 1 — an attribute audit 

import xarray as xr

CHECKS = {
    "data": ["standard_name", "units", "_FillValue", "missing_value", "scale_factor", "valid_range", "valid_min", "cell_methods"],
    "time": ["units", "calendar", "bounds"],
    "lat": ["standard_name", "units", "axis", "bounds"],
}


def cf_audit(path, variable):
    raw = xr.open_dataset(path, decode_cf=False)
    print(f"{path}: Conventions = {raw.attrs.get('Conventions', 'missing')}")
    for role, keys in CHECKS.items():
        name = variable if role == "data" else role
        attrs = raw[name].attrs
        print(f"  {name:4} has   {[key for key in keys if key in attrs]}")
        print(f"  {name:4} lacks {[key for key in keys if key not in attrs]}")


cf_audit("air.mon.mean.nc", "air")
cf_audit("oisst-avhrr-v02r01.20240115.nc", "sst")
air.mon.mean.nc: Conventions = COARDS
  air  has   ['units', 'missing_value', 'scale_factor', 'valid_range']
  air  lacks ['standard_name', '_FillValue', 'valid_min', 'cell_methods']
  time has   ['units']
  time lacks ['calendar', 'bounds']
  lat  has   ['standard_name', 'units', 'axis']
  lat  lacks ['bounds']
oisst-avhrr-v02r01.20240115.nc: Conventions = CF-1.6, ACDD-1.3
  sst  has   ['units', '_FillValue', 'scale_factor', 'valid_min']
  sst  lacks ['standard_name', 'missing_value', 'valid_range', 'cell_methods']
  time has   ['units']
  time lacks ['calendar', 'bounds']
  lat  has   ['units']
  lat  lacks ['standard_name', 'axis', 'bounds']

Opening with decode_cf=False matters: decoding moves _FillValue, scale_factor and the time units out of attrs and into encoding, where an attribute check would not see them.

Example 2 — make the axes recognisable 

import rioxarray  # noqa: F401  registers the .rio accessor
import xarray as xr
from rioxarray.exceptions import MissingSpatialDimensionError

sst = xr.open_dataset("oisst-avhrr-v02r01.20240115.nc")["sst"].squeeze(drop=True)
labelled = sst.assign_coords(
    lat=sst.lat.assign_attrs(standard_name="latitude", axis="Y"),
    lon=sst.lon.assign_attrs(standard_name="longitude", axis="X"),
)
for label, da in [("as shipped", sst), ("with standard_name and axis", labelled)]:
    try:
        print(f"{label}: x={da.rio.x_dim}, y={da.rio.y_dim}")
    except MissingSpatialDimensionError as error:
        print(f"{label}: {type(error).__name__}")
as shipped: MissingSpatialDimensionError
with standard_name and axis: x=lon, y=lat

Example 3 — apply a valid range, in the right units 

import numpy as np
import xarray as xr


def apply_valid_range(da):
    """Mask values outside valid_range or valid_min/valid_max, converting packed limits to data units."""
    scale = da.encoding.get("scale_factor", 1)
    offset = da.encoding.get("add_offset", 0)
    if "valid_range" in da.attrs:
        low, high = da.attrs["valid_range"]
    else:
        low, high = da.attrs.get("valid_min", -np.inf), da.attrs.get("valid_max", np.inf)
    low, high = low * scale + offset, high * scale + offset
    return da.where((da >= low) & (da <= high)), (float(low), float(high))


xr.DataArray(np.array([-5.0, 1.0, 50.0, 999.0], dtype="float32"), dims="x", name="v",
             attrs={"valid_min": np.float32(0), "valid_max": np.float32(100)}).to_netcdf("valid_range_test.nc")
v = xr.open_dataset("valid_range_test.nc")["v"]
masked, limits = apply_valid_range(v)
print("as read:", v.values, " masked:", masked.values, " limits:", limits)

sst = xr.open_dataset("oisst-avhrr-v02r01.20240115.nc")["sst"]
masked, limits = apply_valid_range(sst)
print("OISST attributes", sst.attrs["valid_min"], sst.attrs["valid_max"], "-> limits in degrees", limits)
as read: [ -5.   1.  50. 999.]  masked: [nan  1. 50. nan]  limits: (0.0, 100.0)
OISST attributes -300 4500 -> limits in degrees (-3.0, 45.0)

The NCEP file carries a valid_range of −2000 to 2000 with a scale_factor of 1.0, so its limits need no conversion — and exclude nothing.

Explanation

Why conventions exist at all

NetCDF stores arrays and arbitrary attributes; it says nothing about what an attribute called units must contain. CF supplies that agreement, so that a file written by one modelling centre can be read correctly by software written by another. Its attribute names are what xarray, rioxarray, cf-xarray, CDO and plotting libraries look for.

Why decoding moves attributes into encoding

After xarray has applied scale_factor, masked _FillValue and converted time to dates, those attributes no longer describe the values in memory — the values are already in degrees and dates. xarray moves them to .encoding, which it uses again when writing. Anything it does not apply, such as valid_min, stays in .attrs, even when it no longer matches the decoded values.

Why a missing calendar is not an error

CF defines the absence of a calendar attribute as the standard calendar, so NCEP and OISST times decode as ordinary dates. Model output is different: noleap and 360_day calendars have no equivalent in NumPy's datetime type, so xarray decodes them to cftime objects. That changes which date strings exist, how resampling works and whether the times can be plotted — the subject of fixing cftime and out-of-bounds datetime errors.

Why cell_methods changes interpretation

A cell's value can be an instantaneous sample, a mean over the cell and the time step, a maximum, or a sum. Averaging monthly means into a yearly mean is valid; averaging monthly maxima is not the yearly maximum. cell_methods is where a file says which, and when it is missing the answer has to come from the documentation.

Table of whether rioxarray found the spatial dimensions of OISST and NCEP data with and without CF coordinate attributes.
Tools find axes through attributes or conventional names; data with neither is just an array.

Edge cases or notes

  • _FillValue and missing_value together can block writing: a re-saved NCEP subset raised ValueError: Variable 'air' has conflicting _FillValue (nan) and missing_value.
  • COARDS is a subset of CF. COARDS files usually decode correctly but lack standard names, bounds and cell methods.
  • Attributes survive arithmetic. After converting longitude, NCEP's actual_range still said 0 to 357.5.
  • units is free text to xarray. It does not convert or check units; use a units library if conversions matter.
  • Integer valid_range on a float variable is suspect; check whether it refers to packed or unpacked values.
  • coordinates attributes link auxiliary coordinates, such as a CMIP6 height of 2 m, to a variable.
  • CF compliance checkers exist. The CF checker and IOOS compliance checker report missing and malformed attributes.

FAQ

What are the CF conventions?

The Climate and Forecast metadata conventions: agreed attribute names and meanings for NetCDF files, covering units, standard names, missing values, packing, time calendars, coordinates, cell bounds and cell methods.

Does xarray apply valid_min and valid_max?

No. A test file with a valid range of 0 to 100 read back −5 and 999 unchanged. Apply the range yourself, and convert packed limits to data units first.

What is the difference between the fill value and missing value attributes?

Both mark missing data and xarray masks both as NaN. _FillValue is also the value the NetCDF library writes into unwritten parts of a variable; missing_value is an older convention.

Why can rioxarray not find the x and y dimensions of my NetCDF file?

The coordinates lack attributes it recognises. OISST's latitude and longitude had only long_name and units; adding standard_name or axis attributes, or renaming the dimensions to x and y, fixed it.

What does cell_methods mean?

It records what each value represents. "area: time: mean" means an average over the grid cell and the time step, not a sample at a point and instant.

What happens if a NetCDF file has no calendar attribute?

CF defines that as the standard calendar, and times decode as ordinary dates. Model files often set noleap or 360_day instead, which xarray decodes to cftime objects.