Fixing World Port Index Yes, No and Unknown Columns
On this page
Problem statement
73 of the World Port Index's 109 columns are questions with three answers: Yes, No and Unknown, plus a single space for no answer at all. Converted to booleans the usual way, comparing with "Yes", every Unknown becomes False, and "10% of ports have a container facility" comes out of a column where only one port in seven answered. This page counts the answers across all flag columns, shows how far the Yes-or-not reading distorts a share, converts the flags to pandas' nullable boolean type, and finds which questions are answered at all.
Quick answer
Find the flag columns and count every answer in them:
flags = [c for c in wpi.columns if set(wpi[c].dropna().unique()) <= {"Yes", "No", "Unknown", " "}]
print(f"flag columns: {len(flags)}")
v = wpi[flags].stack().value_counts()
print(v.to_string())
flag columns: 73
Unknown 172194
Yes 74315
No 19981
11421
Across the 73 flag columns there are 172,194 Unknown answers, 74,315 Yes, 19,981 No and 11,421 single spaces. Unknown is more than twice as common as Yes, and No is the rarest real answer: the index records what a port has more often than what it lacks.
Booleans, three states and answered questions
1. As booleans, Unknown becomes No
bad = wpi["Facilities - Container"].eq("Yes")
print(f"container: Yes {bad.sum()}, No {(wpi['Facilities - Container'] == 'No').sum()}, Unknown {(wpi['Facilities - Container'] == 'Unknown').sum()}")
print(f"share with a container facility, Unknown as No: {bad.mean():.1%}; among ports that answer: {bad.sum() / wpi['Facilities - Container'].isin(['Yes', 'No']).sum():.1%}")
container: Yes 379, No 195, Unknown 3233
share with a container facility, Unknown as No: 10.0%; among ports that answer: 66.0%
For container facilities, 379 ports answer Yes, 195 No and 3,233 Unknown. Read as Yes-or-not, 10.0% of ports have one. Among the ports that answered, 66.0% do. Neither is the true share of ports with container cranes, but the first is certainly too low, and it is the one a naive == "Yes" gives.
2. Keep three states
tri = wpi[flags].replace({"Yes": True, "No": False, "Unknown": pd.NA, " ": pd.NA}).astype("boolean")
print(tri.dtypes.value_counts().to_dict())
print(tri["Facilities - Container"].value_counts(dropna=False).to_dict())
{BooleanDtype: 73}
{<NA>: 3233, np.True_: 379, np.False_: 195}
pandas' nullable boolean type holds True, False and missing, so replacing Yes with True, No with False and Unknown and the blank with pd.NA keeps the three states: for containers, 379 True, 195 False and 3,233 missing. Sums and means then skip the missing values, .mean() returns the share among answers, and .sum() the count of Yes.
3. Which flags are answered
known = tri.notna().mean().sort_values()
print(known.head(4).round(3).to_string())
print(known.tail(4).round(3).to_string())
IHO S-130 Sea Area 0.000
IHO S-57 Electronic Navigational Chart 0.000
IHO S-101 Electronic Navigational Chart 0.000
Chemical Holding Tank Disposal 0.003
Pilotage - Compulsory 0.778
Entrance Restriction - Other 0.799
Estimated Time of Arrival Message 0.810
Supplies - Potable Water 0.829
Some flags are almost never answered: the IHO sea area and chart columns are empty, and chemical holding tank disposal is answered for 0.3% of ports. Others are well filled: compulsory pilotage for 77.8%, other entrance restrictions for 79.9%, the arrival-message requirement for 81.0% and potable water for 82.9%. Base an analysis on the well-answered questions.
Ordered categories that are not flags
The data used on this page
The US National Geospatial-Intelligence Agency's World Port Index (Pub. 150), the CSV served by its Maritime Safety Information site, downloaded on 28 October 2026 and saved on this site, because NGA revises the index as port information changes. A US government publication, free to use. Depths and lengths are in metres.
from pathlib import Path
import zipfile
import geopandas as gpd
import numpy as np
import pandas as pd
import requests
Path("data").mkdir(exist_ok=True)
# NGA's World Port Index (Pub. 150) as CSV, with Natural Earth's populated places, countries and land. Saved
# 28 October 2026: NGA revises the index as port information changes.
# Live: https://msi.nga.mil/api/publications/download?type=view&key=16920959/SFH00000/UpdatedPub150.csv
ARCHIVE = "nga-world-port-index-2026-10-28.zip"
if not Path(f"data/{ARCHIVE}").exists():
Path(f"data/{ARCHIVE}").write_bytes(requests.get(f"https://data.spatialworkflow.io/{ARCHIVE}", timeout=600).content)
Z = zipfile.ZipFile(f"data/{ARCHIVE}")
for f in ["ne_10m_populated_places_simple.zip", "ne_110m_admin_0_countries.zip", "ne_50m_land.zip"]:
Path(f"data/{f}").write_bytes(Z.read(f))
wpi = pd.read_csv(Z.open("UpdatedPub150.csv"), encoding="utf-8-sig")
print(f"World Port Index: {len(wpi):,} ports, {wpi.shape[1]} columns")
World Port Index: 3,807 ports, 109 columns
Example 1: ordered categories that are not flags
for c in ["Repairs", "Dry Dock", "Shelter Afforded", "Harbor Size"]:
print(c, wpi[c].value_counts(dropna=False).to_dict())
Repairs {'Limited': 1675, 'Unknown': 1039, 'Moderate': 343, nan: 304, 'Major': 293, 'Emergency Only': 153}
Dry Dock {'Unknown': 2969, 'Small': 339, 'Medium': 277, 'Large': 145, nan: 77}
Shelter Afforded {'Good': 1286, 'Fair': 1188, 'Excellent': 618, 'Poor': 460, ' ': 187, nan: 35, 'Unknown': 33}
Harbor Size {'Very Small': 2109, 'Small': 1021, 'Medium': 376, 'Large': 174, ' ': 127}
Not every text column is a flag. Repairs, dry dock, shelter and harbour size are ordered categories, from Emergency Only to Major, Small to Large, Poor to Excellent, and they carry their own Unknown and blank values: 1,039 ports have Unknown repairs and 304 none. Convert these to ordered categoricals with Unknown as missing, not to booleans.
Internal links
- Depths stored as 0 β the same problem in the measurements
- Container and LNG terminals β shares that need this fix
- Reading the index β the column groups
- Missing values in spatial data β missing values in general