⚙️ How-To Metadata, licensing & provenance Data sources & downloads Metadata, Licensing & Provenance · step 15 of 20 Updated

How to publish a dataset with a citable identifier

Problem statement

A dataset that lives at https://example.org/downloads/current.gpkg cannot be cited. The URL will change, the file behind it already has, and a paper that says "data from example.org" is a dead end in five years.

A persistent identifier fixes both halves of that: it resolves to a landing page that survives the reorganisation, and it points at a specific immutable version rather than whatever is current. DOIs are the common choice for research data, and the free routes — Zenodo, Figshare, an institutional repository — all have APIs you can call from a pipeline.

This guide prepares a deposit properly, uploads it, and handles the part people get wrong: versioning, so that the citation in a paper keeps meaning the same bytes.

Quick answer 

import requests, pathlib, json

BASE, TOKEN = "https://zenodo.org/api", "…"

def create_deposit(metadata):
    r = requests.post(f"{BASE}/deposit/depositions",
                      params={"access_token": TOKEN}, json={"metadata": metadata}, timeout=60)
    r.raise_for_status()
    return r.json()

def upload_file(deposit, path):
    bucket = deposit["links"]["bucket"]
    path = pathlib.Path(path)
    with open(path, "rb") as fh:
        r = requests.put(f"{bucket}/{path.name}", data=fh,
                         params={"access_token": TOKEN}, timeout=1800)
    r.raise_for_status()
    return r.json()

def publish(deposit):
    r = requests.post(deposit["links"]["publish"], params={"access_token": TOKEN}, timeout=120)
    r.raise_for_status()
    return r.json()["doi"]

Test the whole flow against the sandbox first — https://sandbox.zenodo.org/api with a sandbox token. Publishing on the live service is irreversible: files in a published record cannot be changed, only superseded by a new version.

Flow from a packaged dataset through deposit creation, upload, metadata and publication to a resolvable DOI.
Publication is one-way; everything that needs to be right must be right before it.

Step-by-step solution

1. Freeze the version first

A citable identifier points at bytes. Produce the release, compute its content hash and its version identifier, and do not touch it again. Dataset versioning explained covers the numbering.

2. Package the deposit so it stands alone

A deposit should be usable by someone with no context: the data, the metadata record, the licence text, an attribution file, a README that states what one feature is, and — if the data is derived — the lineage record.

3. Fill in the repository metadata properly

Repository metadata is what the DOI resolves to, and it is what search engines index. Title, description, creators with ORCIDs, keywords, the licence, and — the field people skip — the related identifiers linking to the source datasets and to any paper.

4. Reserve the DOI before you publish if you need to cite it

Zenodo can reserve a DOI on an unpublished deposit, so a paper can cite the dataset that is published at the same time as the paper. The reserved DOI is in metadata.prereserve_doi.doi.

5. Use the concept DOI and the version DOI deliberately

Repositories mint two: a concept DOI that always resolves to the latest version, and a version DOI for each release. A paper's methods section should cite the version DOI; a general "the data is here" link should use the concept DOI.

6. Put the DOI back into the dataset's metadata

The identifier is metadata about the dataset, and the dataset's own record should carry it — which means either publishing the record twice, or writing it with the reserved DOI before upload.

7. Record the deposit in the pipeline's lineage

So that the artefact and the published record can be tied together later.

8. Do not publish personal data by accident

A repository deposit is irreversible and widely mirrored. Run the privacy gate before the upload, not after — How to run a privacy check before publishing a spatial dataset.

Stack showing a concept DOI resolving to the latest version above three version DOIs.
Cite the version DOI in a method; link the concept DOI in a footer.

Code examples

Example 1 — build the repository metadata from your own record 

import json, pathlib

def zenodo_metadata(record, version, creators, related=()):
    return {
        "title": record["title"],
        "upload_type": "dataset",
        "description": (
            f"<p>{record['description']}</p>"
            f"<p><b>One feature is:</b> {record['feature_definition']}</p>"
            f"<p><b>Content date:</b> {record['content_date']}</p>"
            f"<p><b>Lineage:</b> {'; '.join(record['lineage'])}</p>"
        ),
        "creators": creators,                      # [{"name": "Last, First", "orcid": "…"}]
        "version": version,
        "license": record["licence"].lower(),      # Zenodo expects a lowercase SPDX id
        "keywords": record.get("keywords", []),
        "related_identifiers": [
            {"identifier": r["id"], "relation": r.get("relation", "isDerivedFrom"),
             "scheme": r.get("scheme", "doi")} for r in related
        ],
        "notes": f"Content SHA-256: {record['content_sha256']}",
    }

Putting the content hash in the notes is a small thing that pays off: a user who downloads the file five years later can check it against the record.

Example 2 — the full deposit, with a reserved DOI 

import requests, pathlib

def deposit_dataset(files, metadata, base=BASE, token=TOKEN, sandbox=False):
    base = "https://sandbox.zenodo.org/api" if sandbox else base
    params = {"access_token": token}

    dep = requests.post(f"{base}/deposit/depositions", params=params,
                        json={"metadata": {**metadata, "prereserve_doi": True}},
                        timeout=60)
    dep.raise_for_status()
    dep = dep.json()
    doi = dep["metadata"]["prereserve_doi"]["doi"]
    print("reserved DOI:", doi)

    for f in files:
        f = pathlib.Path(f)
        with open(f, "rb") as fh:
            r = requests.put(f"{dep['links']['bucket']}/{f.name}", data=fh,
                             params=params, timeout=3600)
        r.raise_for_status()
        print(f"uploaded {f.name} ({f.stat().st_size / 1e6:.1f} MB), "
              f"checksum {r.json()['checksum']}")
    return dep, doi

The bucket API streams the file rather than loading it into memory, which matters as soon as the dataset is more than a few hundred megabytes.

Example 3 — publish a new version of an existing record 

def new_version(concept_record_id, files, metadata, base=BASE, token=TOKEN):
    params = {"access_token": token}
    r = requests.post(f"{base}/deposit/depositions/{concept_record_id}/actions/newversion",
                      params=params, timeout=60)
    r.raise_for_status()
    draft_url = r.json()["links"]["latest_draft"]
    draft = requests.get(draft_url, params=params, timeout=60).json()

    # a new version starts as a copy: remove the old files before uploading the new ones
    for f in requests.get(draft["links"]["files"], params=params, timeout=60).json():
        requests.delete(f"{draft['links']['self']}/files/{f['id']}", params=params, timeout=60)

    for f in files:
        f = pathlib.Path(f)
        with open(f, "rb") as fh:
            requests.put(f"{draft['links']['bucket']}/{f.name}", data=fh,
                         params=params, timeout=3600).raise_for_status()

    requests.put(draft["links"]["self"], params=params,
                 json={"metadata": metadata}, timeout=60).raise_for_status()
    return requests.post(draft["links"]["publish"], params=params, timeout=120).json()["doi"]

The file deletion is the step people miss: a new version inherits the previous version's files, so uploading without clearing them publishes both.

Explanation

Why a DOI is different from a URL

A DOI is an identifier plus a promise of resolution. The registry holds a mapping from the identifier to a current location, and the repository commits to keeping the landing page alive. When the server is replaced, the mapping is updated and every citation keeps working — which a bare URL does not.

Why immutability is the point

The value of a citation is that a reader can obtain the same data. A repository that let you edit files behind a published DOI would destroy that, which is why they do not. The corollary is that the check for personal data, wrong CRS, missing licence or a stale metadata record has to happen before publication, because afterwards the only remedy is a new version and a note.

Why concept and version DOIs both exist

A methods section needs to say exactly which bytes were used, which is the version DOI. A project page wants a link that stays current, which is the concept DOI. Publishing only one of them forces users to choose between reproducibility and freshness.

They are how a reader gets from your derived product to its sources, and how a source's maintainers see what has been built from their work. isDerivedFrom for the inputs, isSupplementTo for a paper, isNewVersionOf for a supersession — three relations cover almost everything.

Checklist of deposit contents: the frozen data, the metadata record, licence and attribution, a README, the lineage record and the content checksum.
A deposit has to stand alone for someone with no context.

Edge cases or notes

  • Test on the sandbox. Publishing is irreversible.
  • File size limits apply. Zenodo's default cap is 50 GB per record with larger sizes on request.
  • Lowercase SPDX ids. Zenodo expects cc-by-4.0, not CC-BY-4.0.
  • Put a README in the zip. The landing page description is not downloaded with the files.
  • ORCIDs make creators unambiguous. Names are not identifiers.
  • Archive the code too. A DOI for the pipeline repository, linked with isSupplementTo.
  • Embargo if you must publish late. Repositories support an embargo date with a licence.
  • Mint the DOI once per version, not per file.

FAQ

How do I get a DOI for a spatial dataset?

Deposit it in a repository that mints them — Zenodo, Figshare or an institutional repository — through its API, with metadata, files and a licence, then publish the deposit.

What is the difference between a concept DOI and a version DOI?

The concept DOI always resolves to the latest version; a version DOI points at one immutable release. Cite the version DOI in a method and link the concept DOI elsewhere.

Can I change the files after publishing?

No. Published records are immutable; you publish a new version instead. That is why the privacy and metadata checks have to run before the upload.

Can I reserve a DOI before publishing?

Yes. Create the deposit with prereserve_doi and read the identifier from the response, so a paper can cite a dataset published at the same time.

What should go in the deposit besides the data?

The metadata record, the licence text, the attribution notices, a README defining what one feature is, and the lineage record for derived data.

How do I publish an update?

Use the repository's new-version action, delete the inherited files, upload the new ones, update the metadata and publish. Forgetting the deletion publishes both versions' files together.