🔧 Fix Biodiversity & species data Data cleaning Biodiversity & Species Data · step 17 Notebook By · Updated

Occurrence records fall in the ocean

On this page

Problem statement

A terrestrial species has records in the Pacific, a cluster at 0°N 0°E in the Gulf of Guinea, and a line of points a kilometre offshore all along a coast. None of those is the species swimming.

They are four different bugs with four different fixes: a missing coordinate written as zero, a transposed or negated sign, a coarse land mask that cannot resolve a coastline, and a genuine georeferencing error. Treating them as one problem — "drop anything not on land" — throws away real coastal records and keeps the ones that matter least.

Quick answer

Classify the offshore records by how far offshore they are:

import io, os, zipfile
import numpy as np, geopandas as gpd, requests
from gbif import search

def natural_earth(name, url):
    path = f"bio/{name}.gpkg"
    if not os.path.exists(path):
        blob = requests.get(url, timeout=300).content
        zipfile.ZipFile(io.BytesIO(blob)).extractall(f"bio/{name}")
        shp = [f for f in os.listdir(f"bio/{name}") if f.endswith(".shp")][0]
        gpd.read_file(f"bio/{name}/{shp}").to_file(path, driver="GPKG")
    return gpd.read_file(path)

land = natural_earth("ne_land",
                     "https://naciscdn.org/naturalearth/110m/physical/ne_110m_land.zip")

lynx = search(taxonKey=2435263, hasCoordinate="true")
pts = gpd.GeoDataFrame(lynx, crs=4326,
                       geometry=gpd.points_from_xy(lynx.decimalLongitude,
                                                   lynx.decimalLatitude))
on_land = gpd.sjoin(pts, land, predicate="within", how="left").index_right.notna().values
sea = pts[~on_land].to_crs(6933)
coast = land.to_crs(6933).boundary.union_all()
offshore_km = sea.distance(coast) / 1000

print(f"{len(pts):,} records, {len(sea):,} outside the 110 m land polygons")
for lo, hi, label in [(0, 5, "within 5 km of the coast"),
                      (5, 50, "5–50 km offshore"),
                      (50, 1e6, "more than 50 km offshore")]:
    sel = (offshore_km >= lo) & (offshore_km < hi)
    print(f"  {label:<28} {sel.sum():>4} records")
zeros = (pts.decimalLatitude.abs() < 0.01) & (pts.decimalLongitude.abs() < 0.01)
print(f"at 0,0 exactly: {int(zeros.sum())}")
6,353 records, 68 outside the 110 m land polygons
  within 5 km of the coast       32 records
  5–50 km offshore               29 records
  more than 50 km offshore        7 records
at 0,0 exactly: 3

The distribution of distances is the diagnosis. A record 500 m offshore is a mask artefact; a record 500 km offshore is an error with a cause worth finding.

Map of North America with the 68 Canada lynx records that fall outside the 1:110m land polygons, coloured by distance offshore: most within a few kilometres of the Atlantic coast of the Maritimes and Alaska, and a handful far out in the ocean. Map of North America with the 68 Canada lynx records that fall outside the 1:110m land polygons, coloured by distance offshore: most within a few kilometres of the Atlantic coast of the Maritimes and Alaska, and a handful far out in the ocean.
The 68 records outside the land polygons, by distance offshore. Most sit on the ragged coast of the Maritimes and Alaska, within a few kilometres — the land polygon is coarse there, not the record. The few red ones far out are errors: one in the Aleutians 715 km from the simplified coast, one in the Arctic Ocean, two in James Bay, and three more at 0,0, off this map.
Causes of offshore occurrence records with the signature and the fix. the distance offshore names the causeWhat it looks likeWhat to doa cluster at exactly 0, 0A missing coordinate written aszero — drop ita mirrored cluster across theequatorA sign error: test against theknown range, not against landa line of points a kilometreoffshoreThe coastline is too coarse — usea finer one or a buffera lone record 500 km out to seaA georeferencing error worthreporting to the publisher spatialworkflow.io
Four causes; the distance offshore tells you which one you have.

Exact zeros, sign errors, a matching land mask

1. Check for exact zeros first

0, 0 is a missing coordinate that a spreadsheet filled in. GBIF flags it as ZERO_COORDINATE; so does a one-line test.

2. Test for sign errors

Negate the longitude, the latitude, or both, and see whether the record lands on plausible habitat inside the species' known range. A cluster that mirrors an existing one across the equator or the prime meridian is diagnostic.

3. Use a land mask that matches your tolerance

A 1:110 m coastline is tens of kilometres from the truth in places. A finer layer does not automatically flag fewer records — it resolves lakes and inlets that the coarse one fills in — so compare the two and look at what moved.

4. Buffer the coast rather than dropping

A small positive buffer on the land polygons keeps genuine coastal records and removes the rest. Choose the buffer to match the mask's resolution, and say what it was.

5. Look at what is left by dataset

Offshore records are rarely random: a handful of datasets produce most of them, and the pattern within a dataset names the cause.

6. Report them rather than deleting them

They are evidence about the publisher's georeferencing and worth sending back.

Coastal records against a coarse and a fine coastline with a buffer applied. most offshore records are the coastline’s faulta coarse coastline5 records "in the sea"a finer coastline + 2 km4 on land, 1 still offshoreThe finer layer flagged more records here, not fewer — it resolves lakes the coarse one fills in spatialworkflow.io
Most offshore records are the coastline's fault, and a buffer sized to the mask keeps them.

Finer coastlines and sign tests

The data used on this page

Occurrence records are pulled from the GBIF occurrence search API by the code on this page. gbif.search is the paging function from How to download GBIF occurrences in Python, saved as a module by the setup. GBIF reprocesses continuously, so the setup answers each request with the reply it got on 9 October 2026; a live pull differs in the last digits.

import hashlib
import io
import os
import zipfile

import requests

# GBIF adds and revises records daily. So that the numbers below reproduce, every GBIF request on
# this page is answered from the reply it got on 9 October 2026 (one file per request, named by a
# SHA-1 of its URL); any other request, or all of them once these lines are deleted, goes live.
SAVED = zipfile.ZipFile(io.BytesIO(requests.get(
    "https://data.spatialworkflow.io/gbif-api-2026-10-09.zip", timeout=300).content))
live_get = requests.get


def saved_get(url, params=None, **kwargs):
    prepared = requests.Request("GET", url, params=params).prepare()
    try:
        body = SAVED.read(hashlib.sha1(prepared.url.encode()).hexdigest()[:16] + ".json")
    except KeyError:
        return live_get(url, params=params, **kwargs)
    reply = requests.models.Response()
    reply.status_code, reply._content, reply.url = 200, body, prepared.url
    reply.headers["Content-Type"] = "application/json"
    return reply


requests.get = saved_get

os.makedirs("bio", exist_ok=True)
with open("gbif.py", "w") as f:
    f.write('''
import pandas as pd, requests

API = "https://api.gbif.org/v1"
MAX_OFFSET = 100_000
FIELDS = ["key", "species", "speciesKey", "decimalLongitude", "decimalLatitude",
          "coordinateUncertaintyInMeters", "year", "month", "basisOfRecord",
          "datasetKey", "institutionCode", "issues"]


def count(**params):
    return requests.get(f"{API}/occurrence/search", params={**params, "limit": 0}, timeout=120).json()["count"]


def search(limit=None, page=300, **params):
    total = count(**params)
    if limit is None and total > MAX_OFFSET:
        raise ValueError(f"{total:,} records exceeds the {MAX_OFFSET:,} offset limit")
    rows, offset = [], 0
    target = total if limit is None else min(limit, total)
    while offset < target:
        body = requests.get(f"{API}/occurrence/search",
                            params={**params, "limit": min(page, target - offset), "offset": offset},
                            timeout=180).json()
        rows += body["results"]
        offset += len(body["results"])
        if body["endOfRecords"] or not body["results"]:
            break
    return pd.DataFrame([{f: r.get(f) for f in FIELDS} for r in rows])
''')
print("gbif.py written")
gbif.py written

Example 1 — a finer coastline changes the count

fine = natural_earth("ne_land_10m",
                     "https://naciscdn.org/naturalearth/10m/physical/ne_10m_land.zip")

for name, layer in [("110 m (coarse)", land), ("10 m (fine)", fine)]:
    hit = gpd.sjoin(pts, layer, predicate="within", how="left").index_right.notna()
    print(f"{name:<16} {(~hit).sum():>5,} records not on land "
          f"({(~hit).mean():.2%})")

buffered = fine.to_crs(6933).buffer(2_000).union_all()
in_buffer = pts.to_crs(6933).within(buffered)
print(f"{'10 m + 2 km buffer':<16} {(~in_buffer).sum():>5,} records not on land "
      f"({(~in_buffer).mean():.2%})")
110 m (coarse)      68 records not on land (1.07%)
10 m (fine)         90 records not on land (1.42%)
10 m + 2 km buffer    44 records not on land (0.69%)

The finer coastline flags more records, not fewer: 90 against 68. That is the opposite of the expected result and it has a cause — the 1:10 m layer resolves large lakes as water, and a lynx photographed on a frozen lake or from a boat is now "not on land". Only the buffer reduces the count, to 44, and it does so by accepting everything within two kilometres of any shoreline. Picking the mask that gives the fewest flags is how a cleaning step becomes a way of not seeing the problem.

Example 2 — testing for sign errors

suspects = pts[~on_land].copy()
variants = {
    "as recorded": (suspects.decimalLongitude, suspects.decimalLatitude),
    "longitude negated": (-suspects.decimalLongitude, suspects.decimalLatitude),
    "latitude negated": (suspects.decimalLongitude, -suspects.decimalLatitude),
    "both negated": (-suspects.decimalLongitude, -suspects.decimalLatitude),
    "swapped": (suspects.decimalLatitude, suspects.decimalLongitude),
}
fine_union = fine.union_all()
for label, (lon, lat) in variants.items():
    moved = gpd.GeoSeries(gpd.points_from_xy(lon, lat), crs=4326)
    print(f"  {label:<20} {moved.within(fine_union).sum():>4} of {len(suspects)} "
          f"would fall on land")
  as recorded            39 of 68 would fall on land
  longitude negated      58 of 68 would fall on land
  latitude negated        6 of 68 would fall on land
  both negated           10 of 68 would fall on land
  swapped                 8 of 68 would fall on land

Negating the longitude puts 58 of the 68 on land — and that result means nothing, because negating a North American longitude lands in Asia, which is mostly land. The test only works against the species' known range, not against land in general: run it, then check whether the rescued records fall inside the range the rest of the data describes. Here they do not, so the sign hypothesis is rejected and the records are simply imprecise or genuinely on water.

Example 3 — which datasets produce them

offshore = pts[~pts.to_crs(6933).within(buffered)].copy()
print(f"{len(offshore)} records more than 2 km from the fine coastline")
by_dataset = offshore.datasetKey.value_counts().head(5)
for key, n in by_dataset.items():
    total = (pts.datasetKey == key).sum()
    print(f"  {key[:8]}… {n:>4} offshore of {total:>5,} records "
          f"({n / total:.1%} of that dataset)")
print(f"\ndatasets contributing offshore records: {offshore.datasetKey.nunique()} "
      f"of {pts.datasetKey.nunique()}")
44 records more than 2 km from the fine coastline
  4bfac3ea…   15 offshore of    61 records (24.6% of that dataset)
  377be098…   14 offshore of 3,281 records (0.4% of that dataset)
  50c9509d…   10 offshore of 1,472 records (0.7% of that dataset)
  bb109a7a…    2 offshore of   490 records (0.4% of that dataset)
  f86a681d…    1 offshore of   148 records (0.7% of that dataset)

datasets contributing offshore records: 7 of 54

Why coarse coastlines mislead

Why coarse coastlines produce so many false positives

A 1:110 m coastline generalises fjords, islands, spits and estuaries away entirely, so any record on one of them is "in the sea". For a country like Canada, with an enormously convoluted coast and thousands of islands, the coarse mask is wrong over huge areas — and a species whose habitat is coastal will lose a disproportionate share of its records to it.

Why the zero coordinate is a special case

0, 0 is in the Gulf of Guinea and is what many systems write when a coordinate is absent. It is not a georeferencing error; it is a missing value with a numeric type. Because it is a legitimate position, no geometric test distinguishes it — it needs its own check, which is why GBIF gives it its own flag.

Why a buffer beats a reprojection

The instinct on finding coastal records offshore is to reproject or re-snap them onto land. That moves data to fit a reference layer, which is backwards: the coastline is the approximation and the record's coordinate is the observation. A buffer accepts records near the mask's boundary without changing them, and leaves the original coordinates intact for whoever has a better coastline.

Offshore record counts by distance band and by land mask. how far offshore, and under which mask110 m mask, total68 of 6,353 (1.07%) within 5 km of the coast32 — mask artefacts 5–50 km offshore29 — imprecise or lake records more than 50 km offshore7 — real errors10 m mask, total90 — more, not fewer10 m mask + 2 km buffer44095 recordsPicking the mask that flags fewest records is how a check becomes a way of not looking spatialworkflow.io
Most of the ocean records are within a few kilometres of the coast, and a finer mask absorbs them.

Country centroids, inland water and marine datasets

  • Country centroids are the land-based equivalent: a whole country's records at one point.
  • Sea-crossing species — seals, seabirds, migrating insects — have legitimate offshore records.
  • Inland water: some land layers exclude large lakes, so lake records look offshore too.
  • Coordinate precision: a record rounded to 0.1° can be 11 km out, which puts coastal records in the sea by construction.
  • COUNTRY_COORDINATE_MISMATCH catches many sign errors before you have to test for them.
  • Marine datasets joined to a terrestrial mask will look catastrophically wrong; check the taxon first.

FAQ

Why are my terrestrial records in the ocean?

Zero coordinates, sign errors, a coarse land mask, or genuine georeferencing errors — and the distance offshore separates them.

Should I just drop everything not on land?

No. Most "offshore" records on a coarse mask are coastal records; a finer coastline or a small buffer keeps them.

How do I detect a sign error?

Negate the longitude, the latitude, and both, and see whether the record lands on plausible habitat.

What is the 0,0 cluster?

Missing coordinates written as zeros. GBIF flags it as ZERO_COORDINATE.

What buffer should I use?

One that matches the coastline's resolution — a couple of kilometres for a 1:10 m layer.

Should I move the records onto land?

No. Record the problem, keep the original coordinate, and report it to the publisher.