🔧 Fix Coastal & marine Movement & time Coastal & Marine · step 20 of 20 Updated

AIS tracks jump across the world

Problem statement

A vessel track drawn from AIS runs from the North Sea to the South Atlantic and back within a minute, or crosses Africa in a straight line, or shows an implied speed of four hundred knots. On a map it is a long straight line through places no ship goes, and it is the single most recognisable AIS data problem.

There are five causes, and they need different fixes: a sentinel position treated as data, a shared or mistyped MMSI, a genuine positional outlier, a coverage gap joined across, and longitudes differenced naively at the antimeridian. Only the third is a data quality problem in the usual sense; the rest are structural.

Quick answer

Filter sentinels first, then flag by implied speed, then split rather than delete:

import numpy as np, pandas as pd
from pyproj import Geod

geod = Geod(ellps="WGS84")

df = df[(df.lat.abs() <= 90) & (df.lon.abs() <= 180)]
df = df[(df.lat != 91.0) & (df.lon != 181.0)]              # AIS "not available"

df = df.sort_values(["mmsi", "timestamp"])
lon, lat = df.lon.values, df.lat.values
_, _, dist = geod.inv(lon[:-1], lat[:-1], lon[1:], lat[1:])
dt = np.diff(df.timestamp.values).astype("timedelta64[s]").astype(float)
same = df.mmsi.values[:-1] == df.mmsi.values[1:]
speed = np.r_[np.nan, np.where(same & (dt > 0), dist / dt * 1.94384, np.nan)]
df["implied_kn"] = speed

print(df.implied_kn.describe(percentiles=[.5, .9, .99, .999]).round(2))

geod.inv computes a geodesic distance, which is correct across the antimeridian and at any latitude. Differencing longitudes and multiplying by 111 km is the other way to produce a jump that is not there.

Triage of five causes of AIS tracks jumping and the fix for each.
Five causes, five fixes; only one of them is an outlier in the usual sense.

Step-by-step solution

1. Remove the sentinel positions

Latitude 91 and longitude 181 mean "position not available". They are valid numbers in the message format, they pass a range check that allows ±91, and they place a vessel beyond the pole.

2. Compute the implied speed geodesically

Great-circle distance between consecutive fixes, divided by the time between them. Guard against zero time differences and against differencing across a change of MMSI.

3. Look at the distribution before choosing a threshold

The 99.9th percentile tells you where the tail starts. A fleet of container ships and a fleet of fast ferries need different numbers, and a threshold picked without looking removes real data or leaves the jumps in.

4. Check whether it is an identity problem

A single MMSI reporting alternately from two distant places at a plausible speed for each is two vessels sharing an identifier, not one vessel teleporting. The test is whether the positions cluster into two coherent tracks.

g = df[df.mmsi == suspect].sort_values("timestamp")
print(g.assign(hour=g.timestamp.dt.floor("h"))
       .groupby("hour")[["lat", "lon"]].mean().round(2).head(24))

5. Distinguish an outlier from a gap

A single bad fix between two good ones produces one jump out and one jump back. A coverage gap produces one long, slow-looking segment. The first is removed; the second is split.

6. Split rather than delete

Deleting a fix joins the two neighbours, which can create a new jump. Splitting the track into segments keeps the uncertainty where it belongs — as a gap in the series.

7. Handle the antimeridian

geod.inv is correct there. A planar difference of longitudes is not: a vessel moving from 179.9°E to 179.9°W moves 0.2°, and the naive arithmetic makes it 359.8°.

Two scenes contrasting a single outlier fix with an out-and-back jump against a coverage gap with one long segment.
An outlier jumps out and back; a gap is a single long leg. They need opposite treatments.

Code examples

Example 1 — classify each jump 

import numpy as np, pandas as pd

def classify_jumps(df, max_knots=40.0):
    out = []
    for mmsi, g in df.sort_values("timestamp").groupby("mmsi", sort=False):
        g = g.copy().reset_index(drop=True)
        fast = g["implied_kn"] > max_knots
        for i in np.flatnonzero(fast.values):
            prev_fast = bool(fast.values[i - 1]) if i > 0 else False
            next_fast = bool(fast.values[i + 1]) if i + 1 < len(g) else False
            gap_s = float(g["gap_s"].values[i]) if "gap_s" in g else np.nan
            kind = ("outlier" if next_fast and not prev_fast
                    else "gap" if gap_s > 3600
                    else "boundary")
            out.append({"mmsi": mmsi, "index": i, "kind": kind,
                        "implied_kn": float(g["implied_kn"].values[i]),
                        "gap_s": gap_s})
    return pd.DataFrame(out)

jumps = classify_jumps(df)
print(jumps["kind"].value_counts().to_string())

An out-and-back pair is an outlier; a single fast step after a long gap is coverage. Classifying before acting is what stops a cleaning rule from deleting real voyages.

Example 2 — remove outliers, split at gaps 

import numpy as np, pandas as pd

def clean_track(g, max_knots=40.0, max_gap_s=3600):
    g = g.sort_values("timestamp").reset_index(drop=True).copy()

    # iteratively drop single fixes that jump out and back
    for _ in range(5):
        fast = (g["implied_kn"] > max_knots).values
        drop = np.flatnonzero(fast[:-1] & fast[1:]) + 0
        if not len(drop):
            break
        g = g.drop(index=drop).reset_index(drop=True)
        g = recompute_kinematics(g)

    g["segment"] = ((g["gap_s"].fillna(np.inf) > max_gap_s)
                    | (g["implied_kn"] > max_knots)).cumsum()
    return g

cleaned = df.groupby("mmsi", group_keys=False).apply(clean_track)
print(f"{len(df):,} → {len(cleaned):,} fixes, "
      f"{cleaned.groupby(['mmsi', 'segment']).ngroups:,} segments")

Five iterations is enough in practice; two consecutive bad fixes need two passes, and more than that is usually an identity problem rather than noise.

Example 3 — detect a shared MMSI 

import numpy as np, pandas as pd
from sklearn.cluster import DBSCAN

def shared_mmsi_score(g, eps_km=200, min_samples=5):
    """Cluster the positions; two dense, well-separated clusters suggest two vessels."""
    xy = np.radians(g[["lat", "lon"]].values)
    db = DBSCAN(eps=eps_km / 6371.0, min_samples=min_samples,
                metric="haversine").fit(xy)
    labels = db.labels_
    clusters = [c for c in set(labels) if c != -1]
    if len(clusters) < 2:
        return {"clusters": len(clusters), "suspect": False}
    sizes = sorted((np.sum(labels == c) for c in clusters), reverse=True)
    return {"clusters": len(clusters), "largest_two": sizes[:2],
            "suspect": sizes[1] / sizes[0] > 0.2}

Two comparably sized, well-separated clusters over a period during which no vessel could travel between them is strong evidence that the MMSI covers more than one transponder.

Explanation

Why sentinels look like data

The AIS message format reserves specific values to mean "not available": latitude 91, longitude 181, speed 102.3 knots, course 360°, heading 511°. All of them are inside the field's numeric range, so a decoder emits them as ordinary numbers and a range check that permits ±91 lets them through. A vessel at latitude 91 plots beyond the pole, which on a Mercator map is a line off the top of the world.

Why the antimeridian produces a fake jump

Longitude differences are only distances after accounting for the wrap. A step from 179.9°E to 179.9°W is 0.2° of longitude, but 179.9 − (−179.9) is 359.8. A planar calculation therefore reports a jump of roughly forty thousand kilometres for a vessel that moved twenty. Using a geodesic function removes the whole class.

Why deleting a bad fix can create a new jump

Removing the offending point joins its neighbours. If the neighbours are far apart in time — because the outlier was the only fix in a quiet period — the new segment can itself exceed the speed threshold, and an iterative filter chases its own tail. Splitting into segments instead records the gap rather than bridging it.

Why a shared MMSI is common enough to check for

MMSIs are entered by hand into the transponder, they are reassigned when a vessel changes flag, and a small number of them are placeholder values that many installations share. A track built by grouping on MMSI alone therefore sometimes interleaves two vessels, and the result is exactly the teleporting pattern — at a plausible speed within each cluster and an impossible one between them.

Two scenes contrasting one coherent vessel track with two separated clusters sharing one MMSI, joined by an impossible leg.
Within each cluster the speeds are plausible; only the leg between them is not.

Edge cases or notes

  • Guard against zero time differences. Duplicate timestamps give infinite speed.
  • Do not difference across a change of MMSI. Group first.
  • Anchored vessels swing. They are not stationary and they are not moving.
  • Helicopters and aircraft appear in some feeds and are genuinely fast.
  • Timestamps may be receiver time. Two receivers can disagree by seconds.
  • Satellite AIS collides in busy areas, producing gaps where traffic is densest.
  • Report every rule's count. A filter that drops 30% has made a decision.
  • Keep the rejected fixes. They are the evidence for the identity checks.

FAQ

Why does my AIS track jump across the world?

Usually a sentinel position — latitude 91 or longitude 181 — treated as data, or a shared MMSI interleaving two vessels. Both look like one ship teleporting.

What speed threshold should I use?

Look at the percentiles of implied speed for your fleet first. Forty knots is above any merchant vessel and below some fast ferries.

Should I delete the bad fixes?

Delete single out-and-back outliers; split at gaps. Deleting a fix that sat alone in a quiet period joins its neighbours and can create a new jump.

Why does a vessel near 180° show a huge jump?

Because the longitude difference was computed planar. Use a geodesic distance function, which is correct across the antimeridian.

How do I tell an outlier from a coverage gap?

An outlier produces two consecutive fast steps — out and back. A gap produces one long step after a long time interval.

Can two vessels share an MMSI?

Yes, through mistyping, reassignment and placeholder values. Cluster the positions: two comparably sized, well-separated clusters are the signature.