πŸ”§ Fix Renewable energy siting Data cleaning Renewable Energy Siting Β· step 17 Notebook By Β· Updated

Solar Timestamps Shifted by an Hour

On this page

Problem statement

The modelled PV output peaks at 13:00 while the sun is highest at 12:20. Or the morning ramp starts an hour before sunrise. Or two irradiance sources that should agree correlate at 0.95 instead of 0.97, and every hourly comparison shows a lopsided scatter. The annual totals look fine, which is why the problem survives: an hour-shifted series has the right amount of energy in the wrong hours.

There are three separate causes, and they stack. Hourly values are averages over an interval that can be labelled at its start, middle or end. Local time can mean civil time with daylight saving, a fixed offset, or something in between. And a solar-position calculation evaluated at the label rather than the middle of the interval puts the sun in the wrong place for every hour.

Quick answer

Diagnose by comparing the irradiance-weighted centre of each day with solar noon. For correctly labelled mid-interval data they match; an hour-ending series is 30 minutes late, a local-time series one or two hours late:

import requests, numpy as np, pandas as pd, pvlib

LAT, LON = 39.86, -4.02

def openmeteo(tz):
    j = requests.get("https://archive-api.open-meteo.com/v1/archive", params={
        "latitude": LAT, "longitude": LON,
        "start_date": "2023-01-01", "end_date": "2023-12-31",
        "hourly": "shortwave_radiation", "timezone": tz}, timeout=120).json()
    s = pd.Series(j["hourly"]["shortwave_radiation"],
                  index=pd.to_datetime(j["hourly"]["time"]))
    return s, j["utc_offset_seconds"]

def lag_minutes(s, assume_utc_offset_h=0):
    """Minutes by which the day's irradiance centre trails solar noon."""
    t = s.index - pd.Timedelta(hours=assume_utc_offset_h)
    t = t.tz_localize("UTC")
    noon = pvlib.solarposition.sun_rise_set_transit_spa(
        pd.DatetimeIndex(sorted(set(t.normalize()))), LAT, LON)["transit"]
    minutes = t.hour * 60 + t.minute
    df = pd.DataFrame({"w": s.to_numpy(), "m": minutes, "day": t.normalize()})
    centre = df.groupby("day").apply(lambda g: (g.w * g.m).sum() / g.w.sum())
    noon_min = noon.dt.hour * 60 + noon.dt.minute + noon.dt.second / 60
    return float(np.nanmedian(centre.to_numpy() - noon_min.to_numpy()))

utc, _ = openmeteo("UTC")
print(f"UTC labels as given: centre trails solar noon by {lag_minutes(utc):+.0f} min")
print(f"UTC, shifted to mid-hour: {lag_minutes(utc.shift(-30, freq='min')):+.0f} min")
UTC labels as given: centre trails solar noon by +30 min
UTC, shifted to mid-hour: -0 min

A lag of about +30 minutes means each value is labelled at the end of its hour. Shifting the labels back by half an hour β€” or evaluating the sun there β€” removes it.

UTC everywhere, label meanings, mid-interval sun

1. Request UTC from every source

Ask each API for UTC explicitly and convert to local time only for display. Every conversion in between is a chance to apply daylight saving twice or not at all.

2. Find out what the label means

Read the documentation, then check it with the solar-noon test above. PVGIS stamps values at ten past the hour and they behave as if centred there; Open-Meteo labels the end of each hour; many station datasets the end; some the start. The test is faster than the documentation and more reliable.

3. Evaluate the sun at the middle of each interval

For hourly means, shift the timestamps by half an interval before computing the solar position. This matters most near sunrise and sunset, where the sun moves fastest in elevation relative to its height.

4. Align two sources by measurement

Before comparing or merging two series, shift one of them by βˆ’2 to +2 hours and keep the offset with the highest correlation. If the best offset is not zero, one of the labels was misread.

5. Check the time zone field in every response

APIs that accept a time zone may apply a single offset to the whole series rather than civil time with daylight saving. The response usually says which offset it used; read it.

One hour of irradiance labelled at its start, its middle and its end, with where the sun must be evaluated. one averaging hour, three possible labels12:00start-of-interval label12:30the middle: evaluate the sun here13:00end-of-interval label (Open-Meteo)15:00the same hour at a fixed UTC+2Solar-noon test: Open-Meteo +30 min, PVGIS 0 min, a fixed-offset "local" series +90 min in January spatialworkflow.io
One interval, three possible labels β€” and the sun has to be evaluated at the middle.

A local-time request that is not civil time

Example 1 β€” a "local time" request that is not civil time

local, offset = openmeteo("Europe/Madrid")
print(f"utc_offset_seconds in the response: {offset} (UTC{offset / 3600:+.0f})")
jan = local["2023-01-10"]
print(f"10 January, 'local' labels: first light {jan[jan > 0].index[0]:%H:%M}, "
      f"peak {jan.idxmax():%H:%M}")
print(f"lag against solar noon if read as UTC{offset / 3600:+.0f}: "
      f"{lag_minutes(local, offset / 3600):+.0f} min")
print(f"lag if read as Madrid civil time (UTC+1 in January): "
      f"{lag_minutes(local['2023-01'], 1):+.0f} min in January")
utc_offset_seconds in the response: 7200 (UTC+2)
10 January, 'local' labels: first light 10:00, peak 15:00
lag against solar noon if read as UTC+2: +30 min
lag if read as Madrid civil time (UTC+1 in January): +90 min in January

When this was run, the archive applied one offset β€” the zone's current one, UTC+2 β€” to the whole year, so January's peak is labelled 15:00. Anyone reading the series as Madrid civil time puts every winter hour one hour late. Requesting UTC avoids the question entirely.

Example 2 β€” what the half hour does to modelled output

j = requests.get("https://archive-api.open-meteo.com/v1/archive", params={
    "latitude": LAT, "longitude": LON, "start_date": "2023-01-01", "end_date": "2023-12-31",
    "hourly": "shortwave_radiation,direct_normal_irradiance,diffuse_radiation",
    "timezone": "UTC"}, timeout=120).json()
w = pd.DataFrame(j["hourly"])
w.index = pd.to_datetime(w.pop("time")).dt.tz_localize("UTC")

def poa(index):
    sun = pvlib.solarposition.get_solarposition(index, LAT, LON)
    return pvlib.irradiance.get_total_irradiance(
        36, 180, sun.apparent_zenith, sun.azimuth, w.direct_normal_irradiance.to_numpy(),
        w.shortwave_radiation.to_numpy(), w.diffuse_radiation.to_numpy(),
        model="isotropic").poa_global.fillna(0)

right = poa(w.index - pd.Timedelta("30min"))
wrong = poa(w.index)
print(f"annual POA: mid-hour {right.sum() / 1000:,.0f}, at label {wrong.sum() / 1000:,.0f} "
      f"kWh/mΒ² ({wrong.sum() / right.sum() - 1:+.1%})")
diff = (wrong.to_numpy() - right.to_numpy())
print(f"hourly error: mean |Ξ”| {np.abs(diff[right.to_numpy() > 0]).mean():.0f} W/mΒ², "
      f"worst {np.abs(diff).max():.0f} W/mΒ²")
annual POA: mid-hour 2,035, at label 2,028 kWh/mΒ² (-0.3%)
hourly error: mean |Ξ”| 33 W/mΒ², worst 91 W/mΒ²

The annual total moves by 0.3%, which nobody would notice; the hours move by 33 W/mΒ² on average and up to 91 W/mΒ² β€” the error that breaks anything that uses the hours: storage sizing, self-consumption, market value. Combine it with the fixed-offset problem above and the hourly error becomes the whole morning or afternoon ramp.

Example 3 β€” aligning PVGIS with Open-Meteo

p = requests.get("https://re.jrc.ec.europa.eu/api/v5_3/seriescalc", params={
    "lat": LAT, "lon": LON, "startyear": 2023, "endyear": 2023,
    "components": 1, "outputformat": "json"}, timeout=120).json()
pv = pd.DataFrame(p["outputs"]["hourly"])
pv.index = pd.to_datetime(pv.pop("time"), format="%Y%m%d:%H%M")
ghi_pvgis = (pv["Gb(i)"] + pv["Gd(i)"] + pv["Gr(i)"])
print(f"PVGIS lag: {lag_minutes(ghi_pvgis):+.0f} min;  "
      f"Open-Meteo lag: {lag_minutes(utc):+.0f} min")
PVGIS lag: -0 min;  Open-Meteo lag: +30 min

PVGIS's lag is zero: its values behave as if centred on their HH:10 stamps, so the sun can be evaluated at the stamp itself. Open-Meteo's is 30 minutes. Joining the two on the hour β€” PVGIS 12:10 floored to 12:00 against Open-Meteo 12:00 β€” pairs values centred 40 minutes apart; shifting Open-Meteo back one hour leaves them 20 minutes apart, which is why that shift wins the correlation test.

Timestamp problems in solar data with the symptom and fix for each. three stacked causessymptomcause β†’ fixcentre +30 min after noonend-of-hour labels β†’ evaluate sunat βˆ’30 mincentre +60–90 minlocal or fixed offset β†’ requestUTCtwo sources best at a 1 h shiftdifferent label conventions β†’align by testhourly errors, totals finesun at the label β†’ mid-intervalpositionsTotals hide every one of these; hourly profiles show them spatialworkflow.io
Three stacked causes; the solar-noon test separates them.

Why totals hide the shift

Why totals hide the problem

Shifting a series in time moves energy between hours without changing the sum. Annual yield, monthly totals and capacity factors come out nearly right, and they are the numbers people check first. Only hourly comparisons, profiles and anything sensitive to when the energy arrives reveal it.

Why the sun position error is not symmetric

At the label of an hour-ending value, the sun is where it was at the end of the hour. In the morning that sun is higher than the hour's average sun, so the model credits the horizontal irradiance to a steeper beam than the one that delivered it; in the evening the reverse. The transposition to a tilted plane amplifies both, which is why the hourly errors are large even when the annual one is modest.

Why a fixed offset is not local time

Civil time in most of Europe switches between two offsets. An API that returns one offset for the whole period is internally consistent β€” it is just not the clock on the wall for half the year. The only representation with no ambiguity is UTC.

Lag of the daily irradiance centre behind solar noon for Open-Meteo, PVGIS and a fixed-offset local series. minutes behind solar noonPVGIS, as stamped0 min β€” centredOpen-Meteo UTC, shifted βˆ’30 min0 minOpen-Meteo UTC, as stamped+30 min β€” hour ending"local" series read as civil time+90 min in January050 min100 minThe solar-noon test reads the convention off the data itself spatialworkflow.io
Zero means the labels are centred; thirty minutes means hour-ending; more means a time zone.

Sub-hourly data and leading labels

  • Sub-hourly data has the same problem at a smaller scale: 15-minute means need a 7.5-minute shift.
  • Instantaneous values (some satellite products, some sensors) need no shift, and treating them as interval means introduces one.
  • Leading and trailing labels can differ between variables in the same dataset; check each.
  • Local solar time is sometimes used in older datasets; it differs from UTC by longitude/15 hours plus the equation of time.
  • pvlib's get_solarposition assumes UTC for naive timestamps; localise explicitly.

FAQ

Why does my PV output peak an hour late?

The irradiance is probably labelled at the end of the hour, or in a local time with daylight saving, and the sun position was evaluated at the label.

How do I check which convention a dataset uses?

Compare the irradiance-weighted centre of each day with solar noon. A lag of about 30 minutes means end-of-interval labels.

Should I use local time or UTC?

UTC for every calculation; local time only for display.

Does the time shift change annual energy?

Very little β€” 0.3% in the Toledo example. It changes hourly values a lot.

What does PVGIS's HH:10 timestamp mean?

It is UTC, and the solar-noon test shows the values centred on it: evaluate the sun at the stamp, with no half-hour shift.

Why does Open-Meteo give a different time zone offset than expected?

When a named time zone is requested, check utc_offset_seconds in the response; the archive may apply one offset to the whole series.