How Many Decimal Places Do Coordinates Need?

Problem statement

A coordinate written as 51.4700 and one written as 51.470022123 describe the same airport, but they make different claims: the first says "within about 10 metres", the second "within a millimetre". How many decimal places to keep, store and publish is a question with a concrete answer once you know what one decimal place of a degree is on the ground, and the answer differs a little between latitude and longitude. Too few decimals moves points; too many claims a precision the data never had.

This page computes the ground size of each decimal place of latitude and longitude at different latitudes on the WGS84 ellipsoid, inspects how many decimals the 80,000 coordinates in OurAirports actually carry, and relates decimal places to the accuracy of the sources that produce coordinates.

Quick answer

Each decimal place is a factor of ten. At the equator: 1 decimal ≈ 11 km, 2 ≈ 1.1 km, 3 ≈ 110 m, 4 ≈ 11 m, 5 ≈ 1.1 m, 6 ≈ 11 cm:

for dec in range(0, 9):
    step = 10.0 ** -dec
    ns = geod.inv(0, 0, 0, step)[2]
    ew45 = geod.inv(0, 45, step, 45)[2]
    print(f"{dec} decimals ({step:.{dec}f}°): {ns:12.4f} m of latitude; {ew45:12.4f} m of longitude at 45°")

0 decimals (1°):  110574.3886 m of latitude;   78846.3347 m of longitude at 45°
1 decimals (0.1°):   11057.4277 m of latitude;    7884.6830 m of longitude at 45°
2 decimals (0.01°):    1105.7428 m of latitude;     788.4684 m of longitude at 45°
3 decimals (0.001°):     110.5743 m of latitude;      78.8468 m of longitude at 45°
4 decimals (0.0001°):      11.0574 m of latitude;       7.8847 m of longitude at 45°
5 decimals (0.00001°):       1.1057 m of latitude;       0.7885 m of longitude at 45°
6 decimals (0.000001°):       0.1106 m of latitude;       0.0788 m of longitude at 45°
7 decimals (0.0000001°):       0.0111 m of latitude;       0.0079 m of longitude at 45°
8 decimals (0.00000001°):       0.0011 m of latitude;       0.0008 m of longitude at 45°

A whole degree of latitude at the equator is 110.6 km; each decimal place divides it by ten, down to 1.1 m at five decimals and 11 cm at six. At 45° a decimal place of longitude is 71% as long: 7.9 m at four decimals.

What datasets carry, decimals by place type, rounding error

1. What a real dataset carries

Count the decimal places as written in OurAirports' CSV:

raw = pd.read_csv("data/airports.csv", dtype=str, keep_default_na=False)
raw = raw[raw.type != "closed"]
dec_lat = raw.latitude_deg.str.split(".").str[1].str.len().fillna(0).astype(int)
dec_lon = raw.longitude_deg.str.split(".").str[1].str.len().fillna(0).astype(int)
print("latitude decimals:", dec_lat.value_counts().sort_index().to_dict())
print(f"median {dec_lat.median():.0f} decimals; places with 10 or more: {(dec_lat >= 10).mean():.1%}")

latitude decimals: {0: 30, 1: 91, 2: 334, 3: 937, 4: 5684, 5: 12930, 6: 38439, 7: 339, 8: 409, 9: 158, 10: 1262, 11: 299, 12: 294, 13: 656, 14: 5439, 15: 5160, 16: 137, 17: 4, 18: 1}
median 6 decimals; places with 10 or more: 18.3%

The median coordinate in OurAirports has six decimals, but 18.3% of latitudes have ten or more, up to eighteen: those digits come from conversions and calculations, not from a measurement to the nanometre.

2. Decimals by type of place

Different contributors and sources produce different precision. Compare large airports with heliports and small airstrips:

raw["dec"] = np.minimum(dec_lat, dec_lon)
print(raw.groupby("type").dec.agg(places="size", median="median", share_4_or_fewer=lambda s: f"{(s <= 4).mean():.0%}").sort_values("places", ascending=False).to_string())

                places  median share_4_or_fewer
type                                           
small_airport    42766     6.0              15%
heliport         23220     6.0              10%
medium_airport    4106     6.0              25%
seaplane_base     1273     6.0              23%
large_airport     1175     6.0              23%
balloonport         63     6.0              14%

The median is six decimals for every type of place. The share with four or fewer, 11 m or coarser, is highest for medium and large airports and seaplane bases, 23–25%.

3. Rounding error at each precision

Round every coordinate and measure how far the points move, to see which precision is safe:

A = raw.astype({"latitude_deg": float, "longitude_deg": float})
for dec in (3, 4, 5, 6):
    e = geod.inv(A.longitude_deg.values, A.latitude_deg.values, A.longitude_deg.round(dec).values, A.latitude_deg.round(dec).values)[2]
    print(f"rounded to {dec} decimals: max displacement {e.max():.3f} m, mean {e.mean():.3f} m")

rounded to 3 decimals: max displacement 78.449 m, mean 37.507 m
rounded to 4 decimals: max displacement 7.843 m, mean 2.836 m
rounded to 5 decimals: max displacement 0.785 m, mean 0.268 m
rounded to 6 decimals: max displacement 0.075 m, mean 0.008 m

Rounding to three decimals moves the average point 37.5 m and the worst 78 m; four decimals, 2.8 m on average; five, 0.27 m; six, 8 mm. Six decimals lose nothing any measurement of these places contains.

Decimals against the source's precision

The data used on this page

Every open place in OurAirports, a public-domain database, read as text so the decimals are counted as published. Ground distances are geodesic, from pyproj.Geod on WGS84.

from pathlib import Path

import numpy as np
import pandas as pd
import requests
from pyproj import Geod

Path("data").mkdir(exist_ok=True)
p = Path("data/airports.csv")
if not p.exists():
    r = requests.get("https://davidmegginson.github.io/ourairports-data/airports.csv", timeout=300)
    r.raise_for_status()
    p.write_bytes(r.content)
geod = Geod(ellps="WGS84")
print(f"{sum(1 for _ in open(p)) - 1:,} rows in the airports file")

86,156 rows in the airports file

The airports file has 86,156 rows, including closed sites, which the steps below leave out.

Example 1: decimals against the precision of the source

Match each precision to the sources that can deliver it:

sources = {"country or city centroid": 10_000, "geocoded street address": 50, "smartphone GPS": 5, "survey-grade GNSS": 0.02}
for name, acc in sources.items():
    dec = int(np.ceil(-np.log10(acc / 111_320)))
    print(f"{name:>26}: accuracy about {acc:>8,} m -> {dec} decimals are enough ({111_320 * 10 ** -dec:,.3f} m)")

  country or city centroid: accuracy about   10,000 m -> 2 decimals are enough (1,113.200 m)
   geocoded street address: accuracy about       50 m -> 4 decimals are enough (11.132 m)
            smartphone GPS: accuracy about        5 m -> 5 decimals are enough (1.113 m)
         survey-grade GNSS: accuracy about     0.02 m -> 7 decimals are enough (0.011 m)

Matching decimals to sources: a centroid needs two decimals, a geocoded address four, a phone's GPS fix five and survey-grade GNSS seven. Writing more decimals than the source supports adds digits, not accuracy.

Each decimal place divides the step by ten

A degree of latitude is about 111 km, so every decimal place divides the step by ten: 0.001° is about 111 m, 0.00001° about 1.1 m. Longitude steps are the same at the equator and shrink with the cosine of the latitude, so at 45° a decimal place of longitude is 71% of one of latitude, and at 60° half. The number of decimals written is therefore a statement about precision: a coordinate's last digit should match the uncertainty of how it was measured.

Most sources fall in a narrow band. Smartphone GPS is typically good to a few metres, so five decimals capture all its information; survey-grade GNSS reaches centimetres, so seven; a geocoded address sits somewhere on a parcel tens of metres wide, so four is honest. Coordinates with ten or more decimals usually come from a computation, a projection or a float conversion, not from a measurement that precise. Six decimals, about 10 cm, is a good default for storage and exchange: it loses nothing a real measurement carries, and it is short.

Trailing zeros, privacy and float32

• Trailing zeros: 51.47 and 51.4700 are the same number; the text precision is lost once a file is parsed into floats.
• Rounding for privacy: dropping decimals deliberately coarsens locations; three decimals hide a home's position within about 100 m.
• Projected coordinates: eastings and northings in metres need two or three decimals for centimetre or millimetre precision.
• Float32: storing as 32-bit floats keeps only about seven significant digits, which near 180° longitude is worse than five decimals.

Internal links

• Fixing coordinates that lose precision when saved to a file — rounding and float32 in practice
• Latitude and longitude explained — the length of a degree
• How to encode coordinates as geohashes in Python — precision as string length

FAQ

How many decimal places should GPS coordinates have?

Five or six: five decimals is about 1 m, six about 10 cm, which covers consumer GPS and leaves margin.

How accurate is 4 decimal places of latitude and longitude?

About 11 m of latitude, and 11 m times the cosine of the latitude of longitude.

Why do some datasets have 15 decimal places?

The values came from a calculation or a float conversion; the extra digits are not real precision.

Does rounding coordinates move them?

Yes, by up to half the last decimal step in each coordinate; rounding OurAirports' places to four decimals moved them by up to 7.8 m.