🔧 Fix Spatial machine learning Spatial Machine Learning · step 13 of 14 Updated

Feature Importance Says Coordinates Are the Best Predictor

Problem statement

You add x and y to a model, the score improves, and the importance table puts them near the top. The physical story you were building — slope drives vegetation, elevation drives temperature — quietly changes.

Measured on a real problem, predicting a vegetation index from terrain:

terrain only    slope 0.459, elevation 0.282, tpi 0.106,
                northness 0.089, eastness 0.065

terrain + x,y   slope 0.395, elevation 0.230, x 0.129, y 0.112,
                northness 0.055, tpi 0.046, eastness 0.033

Coordinates took third and fourth place and pushed TPI from third to sixth. And the model genuinely improved: the true map R² rose from 0.559 to 0.721.

Both things are true at once. The coordinates are informative here and meaningless anywhere else.

Quick answer

Decide from the deployment, not from the importance table:

# predicting inside the sampled area: coordinates help
X = np.column_stack([terrain, coords])        # true map R2 0.721

# predicting anywhere else: they are a lookup table
X = terrain                                    # true map R2 0.559

The same features under block cross-validation, which simulates moving:

terrain only    0.221
terrain + x,y   0.324

A much smaller gain, because a coordinate learned in one place transfers poorly to another.

A model splitting the study area by coordinate to fit local means, which is informative within the area and meaningless outside it.
Coordinates let a tree carve the area into regions with their own means. Powerful here, useless there.

Step-by-step solution

1. Understand what the coordinates are standing in for

A tree given x and y can split the study area into regions and fit a different local mean in each. That captures everything your measured covariates miss: soil, land-use history, microclimate, management.

It is a genuine improvement in predictive accuracy for the sampled area — measured, 0.559 to 0.721 — and it is not a physical explanation.

2. Notice that importance is relative

Adding coordinates did not make slope less important in any physical sense. It made slope's share smaller, because importance sums to one.

TPI fell from 0.106 to 0.046, which means the coordinates absorbed most of what TPI was explaining — the two carry overlapping information about local position.

Importance is a statement about this fitted model on this data, not about the world.

3. Check the transfer cost before keeping them 

                in-area R²    block CV R²
terrain only         0.559          0.221
terrain + x,y        0.721          0.324

The in-area gain is 0.162; the block-CV gain is 0.103. The difference is the part that does not travel.

If the model will only ever predict inside the sampled area, keep them. If it must move, the honest comparison is the block-CV column.

4. Prefer covariates that travel

Replace raw coordinates with variables that encode why a location differs:

  • distance to the coast, to a river, to a city
  • elevation, slope, aspect, topographic position
  • long-term climate normals
  • geology or soil class

These carry the same information where it is causal, and they mean the same thing in a new area. A model using them can transfer; one using x and y cannot.

5. Use permutation importance, on a held-out set

Tree impurity importance is biased towards high-cardinality continuous features — which coordinates are, maximally. Permutation importance on held-out data measures what the model actually uses to predict data it has not seen.

Feature importance shares redistributing when coordinates are added, with TPI falling from 0.106 to 0.046 without becoming less physically relevant.
Importance sums to one, so adding a feature reduces everything else's share by construction.

Code examples

Example 1 — measuring what coordinates cost and buy 

import numpy as np
from sklearn.cluster import KMeans
from sklearn.ensemble import RandomForestRegressor
from sklearn.metrics import r2_score
from sklearn.model_selection import GroupKFold, KFold


def coordinate_effect(X, y, coords, X_holdout=None, y_holdout=None,
                      n_blocks=10):
    """In-area accuracy against transferability, with and without x, y."""
    def model():
        return RandomForestRegressor(n_estimators=200, min_samples_leaf=2,
                                     n_jobs=-1, random_state=0)

    def cv(features, groups=None):
        splitter = (GroupKFold(5).split(features, y, groups) if groups is not None
                    else KFold(5, shuffle=True, random_state=0).split(features))
        predicted = np.empty(len(y))
        for train, test in splitter:
            predicted[test] = model().fit(features[train], y[train]) \
                                     .predict(features[test])
        return r2_score(y, predicted)

    blocks = KMeans(n_blocks, random_state=0, n_init=10).fit_predict(coords)
    with_xy = np.column_stack([X, coords])

    rows = []
    for label, features in (("terrain only", X), ("terrain + x,y", with_xy)):
        random = cv(features)
        spatial = cv(features, groups=blocks)
        true = None
        if X_holdout is not None:
            holdout = (np.column_stack([X_holdout, coords_holdout])
                       if label.endswith("x,y") else X_holdout)
            true = r2_score(y_holdout, model().fit(features, y).predict(holdout))
        rows.append((label, random, spatial, true))
        print(f"  {label:16} random {random:+.3f}  block {spatial:+.3f}"
              + (f"  true {true:+.3f}" if true is not None else ""))

    gain_in = rows[1][1] - rows[0][1]
    gain_out = rows[1][2] - rows[0][2]
    print(f"  coordinates gain {gain_in:+.3f} in-area, {gain_out:+.3f} "
          "under block CV — the difference is what does not travel")
    return rows

Example 2 — permutation importance, which is harder to fool 

import numpy as np
from sklearn.inspection import permutation_importance


def honest_importance(model, X_test, y_test, feature_names, n_repeats=10):
    """Permutation importance on held-out data, with the spread shown."""
    result = permutation_importance(model, X_test, y_test,
                                    n_repeats=n_repeats, random_state=0,
                                    n_jobs=-1)
    order = np.argsort(-result.importances_mean)

    print(f"  permutation importance over {n_repeats} repeats:")
    for i in order:
        mean = result.importances_mean[i]
        std = result.importances_std[i]
        bar = "#" * int(max(mean, 0) * 60)
        print(f"    {feature_names[i]:14} {mean:+.4f} ± {std:.4f}  {bar}")

    coordinate_share = sum(result.importances_mean[i]
                           for i, name in enumerate(feature_names)
                           if name in ("x", "y", "lon", "lat", "easting",
                                       "northing"))
    total = max(result.importances_mean.sum(), 1e-12)
    if coordinate_share / total > 0.2:
        print(f"  ! coordinates account for {coordinate_share / total:.0%} of "
              "the importance — the model will not transfer")
    return result

Permutation importance measures the drop in score when a feature is shuffled, on data the model has not seen. Impurity importance measures how often a feature was split on during training, which favours continuous high-cardinality features regardless of usefulness.

Example 3 — replacing coordinates with covariates that travel 

import numpy as np
from scipy.spatial import cKDTree


def positional_covariates(coords, coast=None, rivers=None, cities=None,
                          elevation=None):
    """Variables that encode why a location differs, not which one it is."""
    features, names = [], []

    for label, targets in (("dist_coast", coast), ("dist_river", rivers),
                           ("dist_city", cities)):
        if targets is None:
            continue
        distances, _ = cKDTree(np.asarray(targets)).query(coords, k=1)
        features.append(distances)
        names.append(label)

    if elevation is not None:
        features.append(np.asarray(elevation))
        names.append("elevation")

    if not features:
        raise ValueError("no positional covariates supplied")

    print(f"  {len(names)} transferable covariates: {names}")
    print("  these mean the same thing in a new area; x and y do not")
    return np.column_stack(features), names

Distance to the coast is 40 km in one place and 40 km in another, and in both it implies a similar maritime influence. An easting of 422,000 implies nothing outside its own UTM zone.

Explanation

Why coordinates are so effective within a study area

Spatial data is autocorrelated: nearby places have similar values. A model that knows where a point is can exploit that directly, without needing any covariate to explain it.

For a tree, x and y are ideal split variables — continuous, high-cardinality, and able to isolate any region. Given enough depth, a tree with coordinates approximates a nearest-neighbour interpolator.

That is why the in-area gain is real and large: 0.559 to 0.721 in the measurement here. It is also why the model has stopped being a model of the process and become a model of the place.

Why impurity importance overstates them

Tree impurity importance counts how much each feature reduced impurity across the splits it was used for. Features with many possible split points get more opportunities.

Continuous coordinates have as many split points as there are distinct values — the maximum. Categorical features with few levels have the fewest.

So impurity importance is biased towards coordinates by construction, before any question of usefulness. Permutation importance on held-out data avoids this, at the cost of being slower.

Why the importance table is not a physical explanation

Three reasons, all of which apply here.

It is relative. Adding a feature reduces every other share, so TPI falling from 0.106 to 0.046 says nothing about TPI's physical relevance.

It is model-specific. A different model on the same data gives a different ranking.

Correlated features share arbitrarily. Coordinates and TPI both encode local position, and the split between them depends on the fitting, not on the world.

Use importance to understand the model. Use domain reasoning and controlled experiments to understand the process.

Why replacing coordinates is worth the effort

A model with coordinates is a very good interpolator for one area. A model with transferable covariates is a weaker interpolator that can be applied elsewhere, checked against new data, and criticised on physical grounds.

The measured cost of dropping coordinates is 0.162 R² in-area. The measured benefit under block CV is that the gap narrows to 0.103 — and in a genuine new region, coordinates are worth nothing at all.

Which trade to make depends entirely on what the model is for, and that is the question the importance table cannot answer.

Impurity importance favouring high-cardinality features against permutation importance measured on held-out data.
Impurity importance is biased towards coordinates before any question of usefulness arises.

Edge cases or notes

  • Importance is relative and sums to one. Adding a feature reduces every other share.
  • Impurity importance favours continuous high-cardinality features — coordinates maximally.
  • Use permutation importance on held-out data.
  • Coordinates help in-area and not elsewhere — 0.162 against 0.103 here.
  • Prefer distance-to-feature covariates that mean the same thing anywhere.
  • Correlated features split importance arbitrarily.
  • A latitude term is not the same as y — it has a physical meaning through insolation.
  • Report the feature list with the model; two models with different features are different models.

FAQ

Why do coordinates dominate my feature importance?

Because they are continuous and high-cardinality, which impurity importance favours, and because spatial autocorrelation makes location genuinely predictive within the study area.

Does that mean my model is wrong?

No. Measured, adding coordinates raised the true in-area R² from 0.559 to 0.721. It means the model has learned this place rather than the process.

Should I remove the coordinates?

If the model must work elsewhere, yes. If it will only ever predict inside the sampled area, keeping them is a legitimate accuracy gain.

What should I use instead?

Covariates that encode why a location differs: distance to coast, river or city, elevation, aspect, climate normals. These mean the same thing in a new area.

Why did another feature's importance drop when I added coordinates?

Importance sums to one, so shares redistribute. TPI fell from 0.106 to 0.046 here without becoming less physically relevant.

Is permutation importance better?

Yes, for this purpose. It measures the score drop when a feature is shuffled on held-out data, rather than counting splits during training.

Can I trust feature importance as an explanation?

Only as an explanation of the model. It is relative, model-specific, and splits correlated features arbitrarily.