Yield monitor data explained: what the numbers really are

Problem statement

A yield map looks like a measurement of the field. It is a measurement of a machine moving through the field, and the difference produces artefacts that dominate the map if they are not removed: a stripe along every start and stop, a halo at every headland turn, a band down every pass where the header was not full, and a smear of about twenty metres wherever the crop changed.

None of these is noise in the statistical sense. They are systematic, they repeat in every year's map from the same machine, and a clustering algorithm will happily build management zones out of them.

This guide explains where each artefact comes from, which of them can be corrected and which can only be removed.

Quick answer

A yield point is a mass flow measured at the top of the elevator, divided by an area inferred from speed and an assumed header width:

yield = mass_flow / (speed × effective_width × moisture_correction)

Every term contributes an error:

The lag, partial widths, start and stop transitions

1. Understand the lag

Grain takes ten to twenty seconds to travel from the header to the mass-flow sensor. At a typical 5 km/h that is 14–28 m, and at 8 km/h it is 22–44 m. Unless the data has been lag-corrected, every value is attributed to ground the machine had already left — which smears every real boundary in the direction of travel.

2. Understand the partial-width problem

The monitor assumes the full header width unless the operator tells it otherwise. On the last pass of a land, around obstacles and on tapered headlands, the header is part full, so the same grain flow is divided by too large an area and the recorded yield is too low — or, where the machine overlaps a previous pass, grain is counted twice over ground already cut.

3. Understand the start and stop transients

Flow builds up over several seconds after the machine starts moving and decays after it stops. Points recorded during either are meaningless, and they cluster at the ends of every pass, which is exactly where headland zones get drawn.

4. Understand the moisture correction

Yield is reported at a standard moisture. If the moisture sensor is out or the correction is applied inconsistently between passes, whole passes shift relative to each other by several percent, producing stripes that follow the machine rather than the field.

5. Understand the position offset

The GNSS antenna is on the cab roof, the crop entered the header several metres ahead. A constant offset moves the entire map, which matters when comparing with a prescription or a soil sample.

6. Know which artefacts can be corrected

• Lag — correctable if the delay is known, by shifting values back along the track.
• Position offset — correctable with a constant translation.
• Moisture — correctable if the raw moisture is recorded.
• Partial width — correctable only if the true swath width was logged; otherwise the affected points have to go.
• Start/stop transients — not correctable; remove them.

7. Expect to remove a substantial fraction of the points

A defensible cleaning pass removes start and stop transients, overlapping passes, extreme speeds, extreme flows and impossible yields. Losing 10–25% of the points is normal, and the number should be reported.

What a raw yield file contains

The data used on this page

Three corn harvests, 2016–2018, of a 22 ha field at Porto Felloni near Ferrara, published on Zenodo (CC BY 4.0) with the paper that used them. Per the paper, the monitor logged one point a second from a combine with a 6 m header, and the authors had already removed border effects and, in 2016, two lines where the combine passed twice. Each file has a record number, easting, northing, yield in kg/ha and grain moisture — no timestamp, speed, swath or flow columns, so time here is the record number.

import os
import zipfile

import geopandas as gpd
import numpy as np
import pandas as pd
import requests

if not os.path.exists("FIELD_YIELD"):
    reply = requests.get("https://zenodo.org/records/14216165/files/FIELD_YIELD.zip?download=1", timeout=300)
    reply.raise_for_status()
    open("FIELD_YIELD.zip", "wb").write(reply.content)
    zipfile.ZipFile("FIELD_YIELD.zip").extractall(".")

files = {2016: "FIELD_YIELD/yield2016/yield2016.gpkg",
         2017: "FIELD_YIELD/yield2017/yield2017.shp",
         2018: "FIELD_YIELD/yield2018/yield2018.shp"}
raw = {year: gpd.read_file(path) for year, path in files.items()}
for year, g in raw.items():
    print(f"{year}: {len(g):,} points, declared CRS {g.crs.to_epsg() or g.crs.name}, "
          f"eastings {g['EST'].min():,.0f}–{g['EST'].max():,.0f}")

2016: 21,169 points, declared CRS 32632, eastings 1,754,313–1,754,862
2017: 18,444 points, declared CRS 32632, eastings 1,754,315–1,754,867
2018: 22,616 points, declared CRS Roma_1940_Gauss_Boaga_Ovest, eastings 1,754,320–1,754,870

Example 1 — what a raw yield file contains

y = raw[2017].drop(columns="geometry").rename(columns={"YIELD": "yield_kg_ha", "HUMID": "moisture_pct"})
y["yield_t_ha"] = y["yield_kg_ha"] / 1000
print(y[["yield_t_ha", "moisture_pct"]].describe(percentiles=[.01, .5, .99]).round(2))

step = np.hypot(y["EST"].diff(), y["NORD"].diff())             # one record a second
y["speed_kmh"] = step * 3.6
print(f"\npoints {len(y):,}")
print(f"zero or negative yield: {(y.yield_t_ha <= 0).sum():,}")
print(f"yield at its maximum, {y.yield_t_ha.max():.1f} t/ha: {(y.yield_t_ha == y.yield_t_ha.max()).sum():,} points")
print(f"median point spacing {step.median():.2f} m = {y.speed_kmh.median():.1f} km/h; "
      f"record-number gaps over 1 s: {(y['NUMBER'].diff() > 1).sum():,}")

       yield_t_ha  moisture_pct
count    18444.00      18444.00
mean        14.49         20.79
std          2.84          1.98
min          0.02          7.20
1%           3.03         17.50
50%         14.97         20.80
99%         18.00         25.46
max         18.00         31.20

points 18,444
zero or negative yield: 0
yield at its maximum, 18.0 t/ha: 1,367 points
median point spacing 1.58 m = 5.7 km/h; record-number gaps over 1 s: 55

Two things in that summary are artefacts, not crop. The file tops out at 18.0 t/ha: 1,367 points, 7.4% of it, sit exactly at that ceiling — a cap applied by the monitor or its software, not 1,367 places where corn yielded exactly 18 t/ha. And the spacing gives the speed — 5.7 km/h — because the record number is a clock.

The declared CRS deserves the same scepticism as the values. The 2016 and 2017 files say they are in UTM zone 32N, but their eastings of 1.75 million metres are impossible in UTM, where eastings stay below a million. They are in the Italian Gauss-Boaga grid, as the 2018 file correctly says:

from pyproj import Transformer

e, n = raw[2017]["EST"].median(), raw[2017]["NORD"].median()
for epsg in (32632, 3003):
    lon, lat = Transformer.from_crs(epsg, 4326, always_xy=True).transform(e, n)
    print(f"read as EPSG:{epsg}: {lat:.4f}° N, {lon:.4f}° E")
fields = {year: g.set_crs(3003, allow_override=True) for year, g in raw.items()}

read as EPSG:32632: 43.6676° N, 24.5545° E
read as EPSG:3003: 44.6944° N, 12.2127° E

Read as UTM, the field lands hundreds of kilometres to the east; read as Gauss-Boaga, it lands at Porto Felloni. The label is the error, and nothing in the file raises it.

Example 2 — reconstruct the geometry of each pass

Without timestamps, a pass boundary shows up as a jump in position or a reversal of direction between consecutive records:

def add_pass_geometry(y, gap_m=15):
    y = y.sort_values("NUMBER").copy()
    dx, dy = y["EST"].diff(), y["NORD"].diff()
    # heading over five records, not one: at 1.6 m spacing, GPS jitter swings a
    # point-to-point heading by tens of degrees
    hx, hy = y["EST"].diff(5), y["NORD"].diff(5)
    y["heading_deg"] = np.degrees(np.arctan2(hx, hy)) % 360
    # the field's working direction, from the steps themselves; a pass ends where the
    # combine reverses along it, jumps, or the record numbers skip
    steps = np.column_stack([hx, hy])[5:]
    axis = np.linalg.svd(steps - steps.mean(0), full_matrices=False)[2][0]
    forward = (hx * axis[0] + hy * axis[1]) > 0
    y["pass_id"] = ((np.hypot(dx, dy) > gap_m) | (forward != forward.shift()) |
                    (y["NUMBER"].diff() > 5)).cumsum()

    sizes = y.groupby("pass_id").size()
    print(f"{len(sizes):,} pieces; {int((sizes >= 50).sum())} passes of 50 points or more hold "
          f"{sizes[sizes >= 50].sum() / len(y):.0%} of the points")

    y["point_in_pass"] = y.groupby("pass_id").cumcount()
    y["from_end"] = y.groupby("pass_id")["point_in_pass"].transform("max") - y["point_in_pass"]
    return y


y = add_pass_geometry(y)
long = y[y.groupby("pass_id")["pass_id"].transform("size") >= 50]
ends = long[(long["point_in_pass"] < 5) | (long["from_end"] < 5)]
middle = long[(long["point_in_pass"] >= 5) & (long["from_end"] >= 5)]
print(f"median yield in the first and last 5 points of a pass: {ends.yield_t_ha.median():.2f} t/ha, "
      f"in the rest: {middle.yield_t_ha.median():.2f} t/ha")

124 pieces; 53 passes of 50 points or more hold 98% of the points
median yield in the first and last 5 points of a pass: 11.73 t/ha, in the rest: 15.05 t/ha

Once the passes are reconstructed, the start and stop transients are the first and last few points of each one — here a median 11.7 t/ha against 15.1 t/ha in the rest of the pass, even in a file whose authors had already trimmed the field borders — and the headland is where the passes turn.

Example 3 — estimate the lag, then correct it

import numpy as np, pandas as pd

def correct_lag(y, lag_s=12.0):
    """Shift each yield value back along its own pass by the sensor delay."""
    out = []
    for pid, g in y.groupby("pass_id", sort=False):
        g = g.sort_values("NUMBER").copy()
        t = (g["NUMBER"] - g["NUMBER"].iloc[0]).values.astype(float)    # seconds, at 1 Hz
        g["yield_lag_corrected"] = np.interp(t, t - lag_s, g["yield_t_ha"].values,
                                             left=np.nan, right=np.nan)
        out.append(g)
    return pd.concat(out)

The delay is a property of the combine, and it can be estimated from the yield data itself. Neighbouring passes driven in opposite directions put the same ground under the sensor at opposite ends of their delay, so they agree best when each is shifted back by the true lag:

from scipy.spatial import cKDTree

def pass_disagreement(y, lag_s):
    c = correct_lag(y, lag_s).dropna(subset=["yield_lag_corrected"])
    tree = cKDTree(c[["EST", "NORD"]].values)
    pairs = tree.query_pairs(r=7.0, output_type="ndarray")        # about one header width apart
    a, b = pairs[:, 0], pairs[:, 1]
    opposite = (c["pass_id"].values[a] != c["pass_id"].values[b]) & (
        np.abs(((c["heading_deg"].values[a] - c["heading_deg"].values[b]) + 180) % 360 - 180) > 150)
    v = c["yield_lag_corrected"].values
    return float(np.median(np.abs(v[a[opposite]] - v[b[opposite]]))), int(opposite.sum())


scores = {lag: pass_disagreement(long, lag) for lag in range(0, 25, 2)}
for lag, (score, n) in scores.items():
    print(f"lag {lag:2d} s: median difference between opposite neighbours {score:.3f} t/ha ({n:,} pairs)")
best = min(scores, key=lambda k: scores[k][0])
corrected = correct_lag(long, best)
moved = (corrected["yield_t_ha"] - corrected["yield_lag_corrected"]).abs()
print(f"best lag {best} s; median change from correcting it: {moved.median():.2f} t/ha")

lag  0 s: median difference between opposite neighbours 2.663 t/ha (14,814 pairs)
lag  2 s: median difference between opposite neighbours 2.610 t/ha (14,615 pairs)
lag  4 s: median difference between opposite neighbours 2.640 t/ha (14,432 pairs)
lag  6 s: median difference between opposite neighbours 2.660 t/ha (14,224 pairs)
lag  8 s: median difference between opposite neighbours 2.635 t/ha (13,999 pairs)
lag 10 s: median difference between opposite neighbours 2.626 t/ha (13,780 pairs)
lag 12 s: median difference between opposite neighbours 2.604 t/ha (13,568 pairs)
lag 14 s: median difference between opposite neighbours 2.660 t/ha (13,345 pairs)
lag 16 s: median difference between opposite neighbours 2.738 t/ha (13,108 pairs)
lag 18 s: median difference between opposite neighbours 2.684 t/ha (12,856 pairs)
lag 20 s: median difference between opposite neighbours 2.672 t/ha (12,605 pairs)
lag 22 s: median difference between opposite neighbours 2.692 t/ha (12,358 pairs)
lag 24 s: median difference between opposite neighbours 2.749 t/ha (12,126 pairs)
best lag 12 s; median change from correcting it: 1.24 t/ha

On this file the curve is flat: the disagreement between opposite neighbours moves by about 0.1 t/ha across the whole range, with no clear minimum, against a spread of several tonnes between neighbouring points. There is no lag left to find — consistent with the monitor having applied its own flow delay, as most do, before the data was exported. Applying a default 12 s correction anyway would have changed the median point by 1.24 t/ha to fix nothing.

That is the case for estimating the lag rather than assuming it: once per combine, from the data, and it needs no trial strip — only a field harvested in alternating directions, which is nearly every field. A flat curve is a valid answer.

Why lag smears boundaries

Why the lag smears boundaries rather than shifting them

If the delay were constant in distance, the map would simply be translated. It is constant in time, so the distance depends on the speed, which varies within and between passes. A boundary is therefore displaced by different amounts in different places, which blurs it rather than moving it — and it is why a lag-uncorrected map cannot resolve a trial strip narrower than about forty metres.

Why partial width is the largest single error

The monitor divides a measured mass by an assumed area. If the header is half full and the monitor assumes it is full, the recorded yield is half the true value — a 50% error, far larger than anything the sensors contribute. It happens on every last pass, around every obstacle and on every tapered headland, which in an irregular field can be a quarter of the points.

Why moisture produces stripes

The reported yield is corrected to a standard moisture, typically 15% for cereals. If the moisture reading drifts or the correction is applied from a manual entry that was changed part way through the day, the correction differs between passes cut at different times. The result is a stripe pattern aligned with the machine's direction rather than with anything in the field.

Why these artefacts survive into management zones

They are spatially structured: headlands are at the edges, partial passes are along one side, start-stop transients are in rows. Clustering finds spatial structure, so a zoning built on uncleaned yield data reproduces the harvest pattern. The give-away is a zone boundary that runs parallel to the direction of travel.

Formats, multiple harvests and calibration

• Formats vary by manufacturer. A yield file is a vendor format with a common shape.
• Multiple harvests in one file. Split by date before anything else.
• Two combines in one field have two lags and two calibrations.
• Calibration drift within a day is common.
• Grain cart weights are the reference. Scale the map to the weighbridge total.
• Dry and wet grain are different products. Record the moisture basis.
• The first year of a new monitor is the worst. Calibration improves.
• Report the cleaning statistics. Removing 20% of points is a result.

Internal links

• How to clean yield monitor data in Python — the cleaning pipeline
• A yield map has impossible values and stripes — the symptoms in detail
• Management zones explained — why uncleaned yield makes bad zones
• How to analyse an on-farm strip trial in Python — where the lag limits resolution
• How to remove spatial outliers in Python — the statistical half of cleaning
• How to clean a GPS track in Python — reconstructing the passes
• GPS error explained — the position component
• How to write a variable-rate prescription map — what the cleaned map feeds

FAQ

What does a yield monitor actually measure?

Mass flow at the top of the clean grain elevator, divided by an area inferred from ground speed and an assumed header width, corrected to a standard moisture.

Why is my yield map smeared along the direction of travel?

Sensor lag. Grain takes ten to twenty seconds to reach the sensor, which at 5–8 km/h is 14–44 m of travel.

Why are there stripes down my field?

Usually partial header width being recorded as full width, or an inconsistent moisture correction between passes cut at different times.

Why is the headland always different?

Start and stop transients, turning, and partial width all concentrate there. Headland points are the least reliable in the file.

How many points should I expect to remove?

Ten to twenty-five per cent is normal for a defensible cleaning pass. Report the number.

Can I fix all of this?

Lag, position offset and moisture are correctable if the raw values were logged. Partial width is correctable only if the true swath was recorded; start and stop transients can only be removed.