How to Batch Process Multiple Shapefiles in Python

On this page

Problem statement

Reprojecting forty shapefiles by hand in a desktop GIS takes an afternoon and goes wrong on file thirty-one. The same job in Python is one loop, whether the operation is:

reprojecting many shapefiles to one CRS
cleaning or standardizing attribute fields
adding calculated columns
filtering features based on an attribute
exporting processed copies to a new folder

The pattern: find the .shp files, run one GeoPandas function on each, and write the results to a separate folder — here on fifteen Natural Earth files, two of them damaged in ways the plain loop writes out without an error.

Quick answer

The safest way to batch process shapefiles in Python is:

define an input folder and output folder
list all shapefiles in the input folder
read each file with GeoPandas
pass the GeoDataFrame into a reusable processing function
save the result with a consistent filename
add basic error handling so one bad file does not stop the whole batch

from pathlib import Path
import geopandas as gpd

input_folder = Path("data/input_shapefiles")
output_folder = Path("data/output_shapefiles")
output_folder.mkdir(parents=True, exist_ok=True)

def process_gdf(gdf):
    gdf = gdf.copy()
    gdf["source"] = "batch_run"
    return gdf

for shp_path in sorted(input_folder.glob("*.shp")):
    gdf = gpd.read_file(shp_path)
    result = process_gdf(gdf)
    output_path = output_folder / f"{shp_path.stem}_processed.shp"
    result.to_file(output_path)

print(len(list(output_folder.glob("*_processed.shp"))), "files written")

15 files written

On the fifteen Natural Earth files this writes fifteen outputs without an error — including the lakes that have no .prj, now copied on without one, and the rivers file cut off mid-copy, whose 6 missing geometries go straight through. Adding a field needs no CRS and no geometry, so nothing checked either. That is the case for the error handling and validation below.

This pattern works for reprojection, attribute updates, filtering, clipping, and other folder-based GIS tasks.

Batch loop with per-file error handling

A batch loop that logs failures and keeps going.

Folders, a loop and one processing function

Once you see the shape, the loop becomes boilerplate you reuse forever.

Set up input and output folders

Use separate folders for source data and processed outputs. This reduces the risk of overwriting your original shapefiles.

from pathlib import Path

input_folder = Path("data/input_shapefiles")
output_folder = Path("data/output_shapefiles")
output_folder.mkdir(parents=True, exist_ok=True)

Writing to a new folder is safer than editing files in place, especially during testing.

Get all shapefiles from the folder

Use pathlib to find only .shp files in the input folder. If you also need subfolders, use rglob("*.shp") instead of glob("*.shp").

shapefiles = sorted(input_folder.glob("*.shp"))      # glob order is whatever the filesystem returns
print(f"Found {len(shapefiles)} shapefiles")
for path in shapefiles:
    print(path.name)

Found 15 shapefiles
lakes.shp
ne_110m_admin_0_boundary_lines_land.shp
ne_110m_admin_0_countries.shp
ne_110m_admin_1_states_provinces.shp
ne_110m_coastline.shp
ne_110m_geography_regions_polys.shp
ne_110m_glaciated_areas.shp
ne_110m_lakes.shp
ne_110m_land.shp
ne_110m_ocean.shp
ne_110m_populated_places.shp
ne_110m_rivers_lake_centerlines.shp
ne_50m_admin_0_countries.shp
ne_50m_populated_places.shp
rivers.shp

This is useful in mixed folders that may also contain .dbf, .shx, .prj, CSV files, or other data.

A shapefile is not just one file. The .shp file must stay with its related sidecar files such as .dbf, .shx, and often .prj.

Read each shapefile with GeoPandas

Loop through the file paths and load each one into a GeoDataFrame.

import geopandas as gpd

for shp_path in shapefiles:
    gdf = gpd.read_file(shp_path)
    print(shp_path.name, len(gdf), "features")

lakes.shp 24 features
ne_110m_admin_0_boundary_lines_land.shp 331 features
ne_110m_admin_0_countries.shp 177 features
ne_110m_admin_1_states_provinces.shp 51 features
ne_110m_coastline.shp 134 features
ne_110m_geography_regions_polys.shp 60 features
ne_110m_glaciated_areas.shp 11 features
ne_110m_lakes.shp 24 features
ne_110m_land.shp 127 features
ne_110m_ocean.shp 2 features
ne_110m_populated_places.shp 243 features
ne_110m_rivers_lake_centerlines.shp 13 features
ne_50m_admin_0_countries.shp 242 features
ne_50m_populated_places.shp 1251 features
rivers.shp 13 features

At this point, gdf is a standard GeoDataFrame, so you can apply any GeoPandas operation you would normally run on a single shapefile.

Apply the same processing logic to each file

The cleanest pattern is to put the repeated GIS logic in a function.

Here is a simple example that adds a new field and reprojects the data to EPSG:4326.

def process_gdf(gdf):
    gdf = gdf.copy()

    if gdf.crs is None:
        raise ValueError("Input shapefile has no CRS defined")

    gdf = gdf.to_crs("EPSG:4326")
    gdf["processed"] = "yes"

    return gdf

This keeps the loop simple and makes it easier to change the processing step later.

For example, if you receive district boundary shapefiles from multiple offices and need them all in WGS84 with one consistent field added, you can use the same function for every file in the folder.

The same structure also works for:

selecting records
renaming columns
calculating areas or lengths
clipping to a boundary
standardizing schemas

Save processed shapefiles to a new folder

Build output names consistently so you can trace results back to the original source file.

written = []
try:
    for shp_path in shapefiles:
        gdf = gpd.read_file(shp_path)
        result = process_gdf(gdf)

        output_path = output_folder / f"{shp_path.stem}_processed.shp"
        result.to_file(output_path)
        written.append(output_path.name)
except ValueError as e:
    print(f"stopped after {len(written)} of {len(shapefiles)} files, at {shp_path.name}: {e}")

stopped after 0 of 15 files, at lakes.shp: Input shapefile has no CRS defined

The try around the whole loop is only here to show where it stopped: with no handling per file, the first bad one ends the run, and every file after it is never attempted.

Using a suffix like _processed avoids name collisions and makes output files easy to identify.

If you write back to shapefile format, remember that shapefiles have limitations such as short field names and weaker support for modern data types.

Add basic progress messages and error handling

For real folders, add status messages and catch errors per file.

failed_files = []

for shp_path in shapefiles:
    print(f"Processing: {shp_path.name}")

    try:
        gdf = gpd.read_file(shp_path)
        result = process_gdf(gdf)

        output_path = output_folder / f"{shp_path.stem}_processed.shp"
        result.to_file(output_path)

        print(f"Saved: {output_path.name}")

    except Exception as e:
        print(f"Failed: {shp_path.name} -> {e}")
        failed_files.append((shp_path.name, str(e)))

print("\nBatch complete")

if failed_files:
    print("Failed files:")
    for name, error in failed_files:
        print(f"- {name}: {error}")

Processing: lakes.shp
Failed: lakes.shp -> Input shapefile has no CRS defined
Processing: ne_110m_admin_0_boundary_lines_land.shp
Saved: ne_110m_admin_0_boundary_lines_land_processed.shp
Processing: ne_110m_admin_0_countries.shp
Saved: ne_110m_admin_0_countries_processed.shp
Processing: ne_110m_admin_1_states_provinces.shp
Saved: ne_110m_admin_1_states_provinces_processed.shp
Processing: ne_110m_coastline.shp
Saved: ne_110m_coastline_processed.shp
Processing: ne_110m_geography_regions_polys.shp
Saved: ne_110m_geography_regions_polys_processed.shp
Processing: ne_110m_glaciated_areas.shp
Saved: ne_110m_glaciated_areas_processed.shp
Processing: ne_110m_lakes.shp
Saved: ne_110m_lakes_processed.shp
Processing: ne_110m_land.shp
Saved: ne_110m_land_processed.shp
Processing: ne_110m_ocean.shp
Saved: ne_110m_ocean_processed.shp
Processing: ne_110m_populated_places.shp
Saved: ne_110m_populated_places_processed.shp
Processing: ne_110m_rivers_lake_centerlines.shp
Saved: ne_110m_rivers_lake_centerlines_processed.shp
Processing: ne_50m_admin_0_countries.shp
Saved: ne_50m_admin_0_countries_processed.shp
Processing: ne_50m_populated_places.shp
Saved: ne_50m_populated_places_processed.shp
Processing: rivers.shp
Saved: rivers_processed.shp

Batch complete
Failed files:
- lakes.shp: Input shapefile has no CRS defined

This is a reasonable minimum for a simple batch workflow, though larger jobs usually benefit from logging and validation checks.

Reprojecting and tagging a folder of shapefiles

The data used on this page

A real folder of shapefiles: thirteen layers of Natural Earth (public domain) at 1:110 million and 1:50 million, downloaded into data/raw/ and then copied flat into data/input_shapefiles/. Two more are copies damaged on purpose in the two ways files most often arrive damaged — lakes.shp without its .prj, and rivers.shp cut off halfway through, as an interrupted copy leaves it. The target CRS is Equal Earth (EPSG:8857).

import io
import shutil
import zipfile
from pathlib import Path

import requests

NE = {"cultural": ["ne_110m_admin_0_countries", "ne_110m_admin_1_states_provinces", "ne_110m_populated_places",
                   "ne_110m_admin_0_boundary_lines_land", "ne_50m_admin_0_countries", "ne_50m_populated_places"],
      "physical": ["ne_110m_coastline", "ne_110m_land", "ne_110m_ocean", "ne_110m_rivers_lake_centerlines",
                   "ne_110m_lakes", "ne_110m_glaciated_areas", "ne_110m_geography_regions_polys"]}
SRC, DST = Path("data/raw"), Path("data/out")
shutil.rmtree(DST, ignore_errors=True)                 # every run of the page starts from nothing
for theme, names in NE.items():
    for name in names:
        if not (SRC / theme / f"{name}.shp").exists():
            scale = name.split("_")[1]
            reply = requests.get(f"https://naciscdn.org/naturalearth/{scale}/{theme}/{name}.zip", timeout=120)
            reply.raise_for_status()
            zipfile.ZipFile(io.BytesIO(reply.content)).extractall(SRC / theme)

# two deliveries damaged on purpose: a missing .prj, and a .shp cut off mid-copy
bad = SRC / "delivered"
bad.mkdir(exist_ok=True)
for ext in ("shp", "shx", "dbf"):
    shutil.copy(SRC / "physical" / f"ne_110m_lakes.{ext}", bad / f"lakes.{ext}")
for ext in ("shp", "shx", "dbf", "prj"):
    shutil.copy(SRC / "physical" / f"ne_110m_rivers_lake_centerlines.{ext}", bad / f"rivers.{ext}")
data = (bad / "rivers.shp").read_bytes()
(bad / "rivers.shp").write_bytes(data[: len(data) // 2])

# the same fifteen, flat in one folder, as most batch scripts expect them
flat = Path("data/input_shapefiles")
shutil.rmtree(flat, ignore_errors=True)
flat.mkdir(parents=True)
for f in SRC.rglob("*"):
    if f.is_file() and f.suffix in (".shp", ".shx", ".dbf", ".prj", ".cpg"):
        shutil.copy(f, flat / f.name)
print(len(sorted(flat.glob("*.shp"))), "shapefiles in", flat)

15 shapefiles in data/input_shapefiles

Only the version that validates says anything about the two damaged files, and only it notices what Web Mercator did to Antarctica.

Example 1: Batch read and write shapefiles from one folder

This example shows the basic read-and-write loop with no extra GIS transformation.

from pathlib import Path
import geopandas as gpd

input_folder = Path("data/input_shapefiles")
output_folder = Path("data/written_shapefiles")
output_folder.mkdir(parents=True, exist_ok=True)

for shp_path in input_folder.glob("*.shp"):
    gdf = gpd.read_file(shp_path)
    output_path = output_folder / shp_path.name
    gdf.to_file(output_path)

Example 2: Batch reproject shapefiles to a target CRS

A realistic task is converting many files to one CRS before later analysis.

from pathlib import Path
import geopandas as gpd

input_folder = Path("data/input_shapefiles")
output_folder = Path("data/reprojected_shapefiles")
output_folder.mkdir(parents=True, exist_ok=True)

target_crs = "EPSG:8857"          # Equal Earth; Natural Earth is already in EPSG:4326

for shp_path in sorted(input_folder.glob("*.shp")):
    gdf = gpd.read_file(shp_path)

    if gdf.crs is None:
        print(f"Skipping {shp_path.name}: missing CRS")
        continue

    gdf = gdf.to_crs(target_crs)
    output_path = output_folder / f"{shp_path.stem}_equal_earth.shp"
    gdf.to_file(output_path)

Skipping lakes.shp: missing CRS

Example 3: Batch add or update an attribute field

This is useful for standardizing datasets before merging or reporting.

from pathlib import Path
import geopandas as gpd

input_folder = Path("data/input_shapefiles")
output_folder = Path("data/standardized_shapefiles")
output_folder.mkdir(parents=True, exist_ok=True)

for shp_path in input_folder.glob("*.shp"):
    gdf = gpd.read_file(shp_path)

    gdf["src_file"] = shp_path.stem

    if "status" in gdf.columns:
        gdf["status"] = gdf["status"].astype(str).str.upper()

    output_path = output_folder / f"{shp_path.stem}_updated.shp"
    gdf.to_file(output_path)

Example 4: Batch process with try/except and progress reporting

This version is safer for larger folders.

from pathlib import Path
import geopandas as gpd

input_folder = Path("data/input_shapefiles")
output_folder = Path("data/output_shapefiles")
output_folder.mkdir(parents=True, exist_ok=True)

failed = []

def process_gdf(gdf, file_name):
    gdf = gdf.copy()
    if gdf.crs is None:
        raise ValueError("Missing CRS")
    if gdf.geometry.isna().any():                        # a truncated .shp reads without error
        raise ValueError(f"{gdf.geometry.isna().sum()} of {len(gdf)} geometries missing")
    gdf = gdf.to_crs("EPSG:3857")
    gdf["src_name"] = file_name
    return gdf

for shp_path in sorted(input_folder.glob("*.shp")):
    try:
        gdf = gpd.read_file(shp_path)
        result = process_gdf(gdf, shp_path.stem)
        output_path = output_folder / f"{shp_path.stem}_processed.shp"
        result.to_file(output_path)
        invalid = (~result.is_valid).sum()
        if invalid:
            print(f"Saved {output_path.name}, but {invalid} geometries are invalid; "
                  f"y reaches {result.total_bounds[1]:,.0f} m")
    except Exception as e:
        failed.append((shp_path.name, str(e)))
        print(f"Error in {shp_path.name}: {e}")

print(f"\nDone. Failed files: {len(failed)}")

Error in lakes.shp: Missing CRS
Saved ne_110m_admin_0_countries_processed.shp, but 1 geometries are invalid; y reaches -242,528,681 m
Saved ne_110m_land_processed.shp, but 1 geometries are invalid; y reaches -242,528,681 m
Error in rivers.shp: 6 of 13 geometries missing

Done. Failed files: 2

Validation is what separates this version from the loop in the quick answer: the lakes and the truncated rivers now fail by name instead of passing through. It also shows something a clean run would hide. Web Mercator cannot show the poles, so Antarctica's coastline is projected to a y of about −242,500 km, and the land and country polygons that touch it come out invalid. The files are written, and nothing raised; only the check said so. For world data, use a projection that has the poles in it.

Why a batch job is a loop plus one function

This approach works because most GIS batch jobs are just file loops plus one reusable transformation.

Globbing anything else either misses datasets or processes each one repeatedly.

The core pieces are:

a folder of input shapefiles
a loop over file paths
a processing function that handles one GeoDataFrame
an output folder for clean results

That makes the workflow easy to test. First, make sure the function works on one shapefile. Then run it across the whole folder.

GeoPandas is usually enough when you need to process a moderate number of vector files with standard operations. For larger pipelines, you may also want:

structured logging instead of print
validation checks before saving
GeoPackage output instead of shapefile
batch processing across subfolders with rglob("*.shp")

If you plan to merge outputs later, consistent CRS and field names matter more than the loop itself.

Missing CRSs, mixed types and accidental overwrites

Some files are missing CRS information

If a file has no CRS, reprojection with to_crs() is not reliable and may fail. You need to know the correct original CRS and assign it first with set_crs().

Mixed geometry types across shapefiles

A batch folder may contain points, lines, and polygons. Some workflows work for all geometry types, but others do not. For example, polygon area calculations do not make sense for point layers.

Field name length limits in shapefiles

Shapefiles have a 10-character field name limit. Longer column names may be truncated when you save output.

Invalid geometries

Spatial operations can fail on broken geometries. If clipping, overlay, or buffering fails, check whether some features are invalid and repair them before running the full batch.

Overwriting outputs accidentally

Avoid writing processed files back into the same folder with the same names. Use a separate destination folder and add a suffix such as _processed.

Internal links

If you need the basics first, see How to Read a Shapefile in Python with GeoPandas.

Related tasks:

If some files fail to load, see How to Read a Shapefile in Python with GeoPandas first to verify that each input file can be opened correctly.

FAQ

How do I loop through all shapefiles in a folder with Python?

Use pathlib and glob:

from pathlib import Path

for shp_path in Path("data/input").glob("*.shp"):
    print(shp_path.name)

This is the standard way to loop through shapefiles in Python for folder-based GIS automation.

Can GeoPandas batch process shapefiles in subfolders too?

Yes. Use rglob("*.shp") instead of glob("*.shp").

from pathlib import Path

for shp_path in Path("data/input").rglob("*.shp"):
    print(shp_path)

This is useful when your data is organized by region, date, or project folder.

Should I overwrite the original shapefiles or save new copies?

Save new copies unless you have a very controlled workflow. A separate output folder is safer and makes it easier to review results.

Is shapefile the best output format for batch GIS workflows?

Not always. Shapefile is still common, but it has field name limits and other restrictions. For many modern workflows, GeoPackage is a better output format, especially when schema consistency matters.

How do I merge all processed shapefiles into one file after the batch?

Collect each GeoDataFrame into a list and combine them with pd.concat, then wrap the result back into a GeoDataFrame:

import pandas as pd, geopandas as gpd
gdfs = [gpd.read_file(p) for p in sorted(output_folder.glob("*_processed.shp"))]
print({str(g.crs) for g in gdfs})
gdfs = [g for g in gdfs if g.crs == "EPSG:3857"]
combined = gpd.GeoDataFrame(pd.concat(gdfs, ignore_index=True), crs=gdfs[0].crs)
print(len(gdfs), "files,", len(combined), "rows,", combined.geom_type.nunique(), "geometry types")

{'EPSG:4326', 'None', 'EPSG:3857'}
13 files, 2666 rows, 5 geometry types

Make sure every input shares the same CRS before concatenating, otherwise coordinates will not line up — pd.concat refuses outright. On this page the check is not academic: the lakes and rivers failed in Example 4, so the outputs the quick answer wrote under the same names were never replaced, and the folder holds files in three CRSs.

How can I speed up reading many shapefiles in a batch?

The read itself is rarely the bottleneck; writing shapefiles and running per-feature geometry operations usually dominate. If you have many independent files, process them in parallel with concurrent.futures.ProcessPoolExecutor, and write to GeoPackage instead of shapefile to avoid sidecar-file overhead.

Can I batch process GeoJSON or GeoPackage files instead of shapefiles?

Yes. The same loop works for any format gpd.read_file() supports; just change the glob pattern, for example input_folder.glob("*.geojson") or input_folder.glob("*.gpkg"). GeoPackage avoids the shapefile field-name and type limitations on output.

How do I skip files that are already processed when re-running the batch?

Check whether the output path exists before processing and continue if it does:

for shp_path in sorted(input_folder.glob("*.shp")):
    output_path = output_folder / f"{shp_path.stem}_processed.gpkg"
    if output_path.exists():
        continue
    ...

This makes the batch resumable after a failure without redoing completed work.