GeoPandas to_file() Fails on a Column Type: How to Fix It #

Problem statement #

The analysis finished, the frame looks right, and the write fails:

ValueError: Invalid field type 

RuntimeError: Failed to write record: ... unsupported field type

pyogrio.errors.FieldError: Could not add field 'tags' of type object

Or it succeeds and quietly changes your data — a timezone stripped, a boolean stored as 0/1, an integer written as a float, a column name silently shortened. GeoPandas holds pandas dtypes, which are rich; file formats hold a small, fixed set of field types. Writing means mapping one onto the other, and anything that does not map has to be converted, dropped, or refused.

Common causes:

• an object column containing lists, dicts, sets or arbitrary Python objects
• a column of mixed types after a merge (int in some rows, str in others)
• datetime64[ns, tz] with a timezone, which shapefiles cannot store
• pandas nullable dtypes (Int64, boolean, string) that a driver does not recognise
• a second geometry column left over from a spatial join
• Decimal, numpy.timedelta64, or bytes values
• NaN in an integer column, forcing a float representation

Quick answer #

Before writing, normalise the schema:

1. print gdf.dtypes and the Python types inside every object column
2. serialise structured values (lists, dicts) to JSON strings
3. cast nullable dtypes to plain float, str or int — or to the driver's supported types
4. drop or convert extra geometry columns
5. choose a format that supports what you have: GeoPackage over shapefile, Parquet for everything

import json
import geopandas as gpd
import pandas as pd

def prepare_for_write(gdf: gpd.GeoDataFrame) -> gpd.GeoDataFrame:
    out = gdf.copy()

    # one geometry column only
    extra_geoms = [c for c in out.columns
                   if c != out.geometry.name and out[c].dtype.name == "geometry"]
    out = out.drop(columns=extra_geoms)

    for col in out.columns:
        if col == out.geometry.name:
            continue
        s = out[col]
        if isinstance(s.dtype, pd.CategoricalDtype):
            out[col] = s.astype(str)
        elif s.dtype == object:
            sample = s.dropna()
            if not sample.empty and isinstance(sample.iloc[0], (list, dict, tuple, set)):
                out[col] = s.map(lambda v: json.dumps(v, default=str) if v is not None else None)
            else:
                out[col] = s.astype("string").astype(object)
        elif str(s.dtype).startswith(("Int", "UInt", "Float", "boolean", "string")):
            out[col] = s.astype(object).where(s.notna(), None)
    return out

prepare_for_write(gdf).to_file("data/out/result.gpkg", driver="GPKG")

Running this once before every write turns a class of driver errors into a predictable conversion you control.

What each format can store #

Step-by-step solution #

Find the offending column #

The error rarely names the column, but a quick inspection does.

import pandas as pd

print(gdf.dtypes)

for col in gdf.columns:
    if col == gdf.geometry.name:
        continue
    if gdf[col].dtype == object:
        types = gdf[col].dropna().map(type).value_counts()
        print(f"\n{col}:")
        print(types.head())
        if len(types) > 1:
            print("  MIXED TYPES — this is very likely the problem")

An object column whose contents are all str is fine. One containing list, dict, Timestamp or a mixture is what the driver rejects.

Serialise structured values #

Lists and dicts have no field type in OGR. Store them as JSON text and parse on the way back in.

import json

gdf["tags"] = gdf["tags"].map(
    lambda v: json.dumps(v, ensure_ascii=False) if v is not None else None
)

# reading back
gdf["tags"] = gdf["tags"].map(lambda s: json.loads(s) if isinstance(s, str) and s else None)

GeoPackage and GeoJSON both handle long strings, so this is lossless. Shapefile truncates text fields at 254 characters, which will silently cut a long JSON payload in half — one more reason not to write shapefiles.

Fix mixed-type columns #

A merge that fills missing rows, or a CSV read without dtypes, easily produces a column holding both numbers and strings.

import pandas as pd

col = "zone_code"
mixed = gdf[col].dropna().map(type).value_counts()
print(mixed)

# decide one representation and enforce it
gdf[col] = pd.to_numeric(gdf[col], errors="coerce")        # everything numeric, bad values → NaN
# or
gdf[col] = gdf[col].astype("string").fillna("")            # everything text

Choosing deliberately matters: coercing to numeric turns "12A" into NaN, which may be a silent data loss you would rather see as an explicit report.

Convert datetimes to what the format supports #

import pandas as pd

# GeoPackage: DateTime is supported; drop the timezone or convert to UTC first
gdf["surveyed_at"] = pd.to_datetime(gdf["surveyed_at"], utc=True).dt.tz_convert(None)

# Shapefile: Date only, no time component
gdf["surveyed_on"] = pd.to_datetime(gdf["surveyed_at"]).dt.date

# Safest across everything: ISO 8601 strings
gdf["surveyed_iso"] = pd.to_datetime(gdf["surveyed_at"], utc=True).dt.strftime("%Y-%m-%dT%H:%M:%SZ")

The dBase table behind a shapefile has a date field but no datetime field, so times are dropped without a warning. Converting explicitly is how you notice.

Handle nullable dtypes #

Pandas' nullable extension types (Int64, boolean, string) are not universally understood by the writers.

import pandas as pd

nullable = [c for c in gdf.columns
            if str(gdf[c].dtype) in {"Int64", "Int32", "boolean", "string", "Float64"}]
print("nullable dtypes:", nullable)

for col in nullable:
    s = gdf[col]
    if str(s.dtype).startswith(("Int", "UInt")):
        gdf[col] = s.astype("float64")           # NaN-capable; ints round-trip exactly
    elif str(s.dtype) == "boolean":
        gdf[col] = s.astype(object).where(s.notna(), None)
    else:
        gdf[col] = s.astype(object).where(s.notna(), None)

Shapefiles have no boolean field at all — a bool becomes an integer or a one-character string depending on the driver version, so convert it yourself and document the choice.

Drop extra geometry columns #

sjoin and merge between two GeoDataFrames leave a second geometry column, which no format can store.

geom_cols = [c for c in gdf.columns if gdf[c].dtype.name == "geometry"]
print("geometry columns:", geom_cols, "active:", gdf.geometry.name)

gdf = gdf.drop(columns=[c for c in geom_cols if c != gdf.geometry.name])

If you need both, write them as WKT text or save two layers in one GeoPackage.

Choose a format that fits the data #

gdf.to_file("data/out/result.gpkg", layer="parcels", driver="GPKG")   # everyday choice
gdf.to_parquet("data/out/result.parquet")                             # keeps pandas dtypes

GeoParquet preserves nullable dtypes, timezone-aware timestamps and long strings, which makes it the right intermediate format inside a pipeline even when the final deliverable is something else.

Code examples #

Example 1: a schema report before the write #

import geopandas as gpd
import pandas as pd

SAFE = {"int64", "int32", "float64", "float32", "bool", "object", "datetime64[ns]"}

def schema_report(gdf: gpd.GeoDataFrame) -> pd.DataFrame:
    rows = []
    for col in gdf.columns:
        s = gdf[col]
        if col == gdf.geometry.name:
            rows.append({"column": col, "dtype": "geometry", "python_types": "-", "issue": ""})
            continue
        types = sorted({type(v).__name__ for v in s.dropna().head(1000)})
        issue = ""
        if len(types) > 1:
            issue = "mixed python types"
        elif types and types[0] in {"list", "dict", "set", "tuple"}:
            issue = "structured value — serialise to JSON"
        elif str(s.dtype) not in SAFE:
            issue = f"dtype {s.dtype} may not be supported by all drivers"
        if len(col) > 10:
            issue = (issue + "; " if issue else "") + "name >10 chars (shapefile truncates)"
        rows.append({"column": col, "dtype": str(s.dtype),
                     "python_types": ",".join(types) or "-", "issue": issue})
    return pd.DataFrame(rows)

report = schema_report(gdf)
print(report.to_string(index=False))
if (report["issue"] != "").any():
    print("\nfix the rows above before writing")

Example 2: a round-trip-safe writer #

import json
from pathlib import Path
import geopandas as gpd
import pandas as pd

JSON_SUFFIX = "_json"

def write_gpkg(gdf: gpd.GeoDataFrame, path, layer="data") -> dict:
    """Write a GeoPackage, recording every conversion applied."""
    out, changes = gdf.copy(), {}

    for col in list(out.columns):
        if col == out.geometry.name:
            continue
        s = out[col]

        if s.dtype.name == "geometry":
            out = out.drop(columns=[col]); changes[col] = "dropped extra geometry"; continue

        sample = s.dropna()
        if not sample.empty and isinstance(sample.iloc[0], (list, dict, tuple, set)):
            out[col + JSON_SUFFIX] = s.map(
                lambda v: json.dumps(v, default=str) if v is not None else None)
            out = out.drop(columns=[col]); changes[col] = "serialised to JSON"; continue

        if isinstance(s.dtype, pd.DatetimeTZDtype):
            out[col] = s.dt.tz_convert("UTC").dt.tz_localize(None)
            changes[col] = "tz → UTC naive"; continue

        if str(s.dtype).startswith(("Int", "UInt", "Float")):
            out[col] = s.astype("float64"); changes[col] = f"{s.dtype} → float64"; continue

        if str(s.dtype) in ("boolean", "string") or isinstance(s.dtype, pd.CategoricalDtype):
            out[col] = s.astype(object).where(s.notna(), None)
            changes[col] = f"{s.dtype} → object"

    Path(path).parent.mkdir(parents=True, exist_ok=True)
    out.to_file(path, layer=layer, driver="GPKG")
    return changes

for col, what in write_gpkg(gdf, "data/out/result.gpkg").items():
    print(f"{col}: {what}")

Example 3: verify the round trip #

import geopandas as gpd

original = gdf
written = gpd.read_file("data/out/result.gpkg")

print("rows      :", len(original), "→", len(written))
print("columns   :", set(original.columns) - set(written.columns), "lost")
print("dtypes    :")
for col in set(original.columns) & set(written.columns):
    if str(original[col].dtype) != str(written[col].dtype):
        print(f"  {col}: {original[col].dtype} → {written[col].dtype}")

Reading back what you wrote is the only way to see what a driver actually did with your schema — the write itself will not tell you.

Example 4: a fallback that writes what it can #

def to_file_lenient(gdf, path, driver="GPKG", **kwargs):
    try:
        gdf.to_file(path, driver=driver, **kwargs)
        return []
    except Exception as first:
        dropped = []
        slim = gdf.copy()
        for col in list(slim.columns):
            if col == slim.geometry.name:
                continue
            try:
                slim[[col, slim.geometry.name]].to_file("/vsimem/probe.gpkg", driver="GPKG")
            except Exception:
                slim = slim.drop(columns=[col]); dropped.append(col)
        if not dropped:
            raise first
        slim.to_file(path, driver=driver, **kwargs)
        return dropped

dropped = to_file_lenient(gdf, "data/out/result.gpkg")
if dropped:
    print("wrote without these columns:", dropped)

Use this in a batch where finishing matters more than completeness — and always report what was dropped.

Explanation #

A GeoDataFrame is a pandas DataFrame with a geometry column, so its columns carry pandas dtypes: NumPy types, extension types like Int64 and string, categoricals, timezone-aware datetimes, and the catch-all object, which can hold literally any Python value. A file format has nothing of the kind. OGR defines a small set of field types — integer, integer64, real, string, date, datetime, binary, and list variants that most drivers do not support — and each driver supports a subset of those.

Writing therefore means mapping a rich type system onto a poor one. Where the mapping is obvious — int64 to Integer64, float64 to Real, str to String — it happens silently. Where it is not, one of two things occurs: the driver raises, which is the good case because you find out immediately; or it makes a lossy choice, which is the bad case because your data changes without a message. A boolean written to a shapefile, a timezone dropped from a timestamp, and a 300-character string truncated to 254 are all in the second category.

object columns are the biggest source of surprises, because pandas uses that dtype for anything it cannot represent natively. A column of strings is object; so is a column of lists, a column of dicts, and a column that got mixed types from a merge. The dtype alone tells you nothing, which is why the useful check is on the Python types inside the column rather than on the dtype itself.

The strategic answer is to raise the ceiling. Shapefile is the most constrained format in common use — 10-character field names, 254-character strings, no booleans, no datetimes, one geometry type per file — and most of these errors are really "you are writing a shapefile". GeoPackage removes almost all of those limits, and GeoParquet removes the rest while preserving pandas dtypes exactly. Converting once at the pipeline boundary is far cheaper than teaching every step to work around the oldest format in the stack.

Edge cases or notes #

• NaN in an integer column: NumPy integers cannot hold NaN, so pandas promotes to float and the file gets Real. Use Int64 in memory and cast to float on write, or fill the gaps.
• Field names over 10 characters: Shapefiles truncate and de-duplicate them (population → populatio). Rename before writing, so you choose the short names.
• pyogrio and fiona differ: Error messages and some type handling are not identical. Note which engine you use when comparing behaviour.
• GeoJSON numbers are doubles: Very large 64-bit integers lose precision. Write them as strings if exactness matters.
• Categoricals: Most drivers do not preserve categories; the values are written as text and read back as plain strings.
• bytes columns: GeoPackage supports a binary field type; shapefiles do not. Base64-encode if you must use the older format.
• Empty frames: Writing a GeoDataFrame with zero rows can fail or produce a layer with no schema. Guard with a row-count check.

Internal links #

• Why Are My Shapefile Column Names Truncated? How to Fix It
• How to Clean and Normalise Attribute Columns in a GeoDataFrame
• How to Read and Write GeoPackage Files in Python
• How to Validate a GeoDataFrame Against a Schema Before Analysis
• Mixed Geometry Types Error When Saving a GeoDataFrame: How to Fix It
• Garbled Attribute Text and UnicodeDecodeError from a Shapefile: How to Fix Encoding

FAQ #

Which column is causing the error? #

Inspect the Python types inside each object column: gdf[col].dropna().map(type).value_counts(). Columns holding lists, dicts, or a mixture of types are the usual culprits, and the dtype alone will not reveal them.

How do I store a list or dict attribute? #

Serialise it to a JSON string with json.dumps() and parse it back after reading. GeoPackage and GeoJSON handle long strings well; shapefiles truncate at 254 characters and will corrupt the payload.

Why did my timezone disappear? #

Shapefiles have no datetime field at all, and several drivers store naive datetimes only. Convert to UTC and drop the timezone deliberately, or write an ISO 8601 string.

Can I write pandas nullable dtypes like Int64? #

Not reliably across drivers. Cast to float64 (which round-trips integer values exactly up to 2^53) or to object with None for missing values before writing.

Why does my write fail after a spatial join? #

sjoin can leave a second geometry column, and no format stores two geometries in one layer. Drop the extra column, or convert it to WKT text.

Should I still be writing shapefiles? #

Only when a recipient requires them. GeoPackage removes the field-name, string-length, boolean, datetime and single-geometry-type limits at no cost, and GeoParquet preserves pandas dtypes exactly.

How do I know what a write actually changed? #

Read the file back and compare dtypes and column sets against the original frame. Drivers convert silently, so a round-trip check is the only reliable audit.