Manifest format

Every dataset directory contains a manifest.json. It is the only input to this documentation site: every catalog page is generated from it.

Why it exists

The documentation build must never open a GeoJSON file.

Parsing a 70 MB file on every build would make mkdocs serve unusable for editing, slow CI substantially, and force CI to check out data it otherwise does not need. Instead, a separate script scans the data when the data changes and writes a few kilobytes of metadata beside it. The docs build reads only that.

This is what makes the catalog scale to hundreds of countries at constant build cost, and what lets CI check out the repository without any .geojson files at all.

Example

{
  "schema_version": 1,
  "body": "earth",
  "iso_a3": "CHL",
  "iso_a2": "CL",
  "m49_region": "South America",
  "name": { "en": "Chile", "es": "Chile" },
  "crs": { "authority": "OGC", "code": "CRS84", "epsg": 4326 },
  "source": {
    "name": "IDE Chile / SUBDERE — División Política Administrativa 2023",
    "url": "https://www.geoportal.cl/",
    "license": "CC-BY-4.0",
    "retrieved": "2026-08-11"
  },
  "status": "ok",
  "notes": "345 communes, not the 346 in the official register: Antártica (12202) is absent because the DPA package excludes Chile's Antarctic claim.",
  "datasets": [
    {
      "level": "ADM1",
      "path": "data/earth/CHL/CHL_ADM1.geojson",
      "preview": "data/earth/CHL/preview/CHL_ADM1.preview.geojson",
      "bytes": 4814221,
      "preview_bytes": 226499,
      "sha256": "5cf4e9d8d34822d4…",
      "features": 16,
      "bbox": [-109.449861, -56.525107, -66.416176, -17.498399],
      "geometry_types": { "MultiPolygon": 10, "Polygon": 6 },
      "properties": ["shapeName", "shapeISO", "shapeGroup", "shapeType"],
      "simplification": { "method": "visvalingam", "tolerance_m": 100 }
    }
  ]
}

Split levels

The municipal level is split by ADM1 parent, so its entry carries a parts array instead of standing on a single file:

{
  "level": "ADM3",
  "split_by": "ADM1",
  "features": 345,
  "path": "data/earth/CHL/CHL_ADM3.geojson",
  "parts": [
    {
      "code": "CL-RM",
      "path": "data/earth/CHL/ADM3/CL-RM.geojson",
      "features": 52,
      "bytes": 164329,
      "sha256": "…",
      "bbox": [-71.72, -34.30, -70.02, -32.92]
    }
  ]
}

• features on the entry is the whole level, so the catalog can always report a total whether or not a combined file exists.
• path is the optional whole-country file, present only when it fits under 20 MB. Its absence is normal and the catalog says so.
• code is the ADM1 parent's ISO 3166-2 code, or a name slug when no ISO code is known.

CI checks that the parts sum exactly to the level's feature count — that is how a split that lost or duplicated a municipality gets caught.

Simplification

Every dataset records what was done to it:

"simplification": { "method": "visvalingam", "tolerance_m": 100 }

tolerance_m is a ground distance, not a percentage. That is deliberate: a percentage keeps a fixed share of each file's vertices, so the resulting resolution depends on how densely the source happened to be digitised and two neighbouring countries end up at different fidelities. A distance gives the whole repository one consistent real-world resolution.

The standard tolerance is 100 m. Chile's 16 regions — one of the world's most complex coastlines — measure 49.7 MB at 10 m, 10.0 MB at 50 m, 4.6 MB at 100 m and 1.6 MB at 250 m. A dataset that would still exceed the size ceiling at the standard tolerance gets a coarser one, and the value recorded here is always the value actually applied.

Hand-authored versus generated

This split is the important part of the format.

Field	Owner	Notes
schema_version	hand	Bumped only on a breaking format change
body, iso_a3, iso_a2, m49_region	hand	Identity
name	hand	Per-locale display names
crs	hand	Authoritative — the files cannot declare this themselves
source	hand	Required. Provenance and licence
status	hand	ok, review or deprecated
notes	hand	Known gaps, quirks, vintage mismatches
datasets[]	machine	Regenerated by build_manifest.py

scripts/build_manifest.py replaces the datasets array wholesale and leaves every other key untouched. Curated metadata therefore survives regeneration — which is what makes it safe to re-run the scanner routinely.

Regenerating

python scripts/build_manifest.py data/earth/CHL

The scanner streams each file with ijson in constant memory, so a 70 MB input costs a few seconds and a few tens of megabytes of RAM rather than a gigabyte-plus of parsed Python objects.

It computes features, bbox, geometry_types, properties, bytes and sha256 from the file itself. Do not hand-edit those fields; CI regenerates the manifest and fails the build if the committed version disagrees.

Checksums and line endings

sha256 is over the raw bytes of the file as stored with LF line endings.

This matters more than it sounds. Git normalises line endings on checkout, so the same file checked out on Windows with CRLF is one byte per line larger — 1.8 MB larger, for Chile's communes file — and hashes to something completely different.

The repository's .gitattributes pins *.geojson to eol=lf so that hashes computed on Windows, on Linux CI and by raw.githubusercontent.com all agree. If you get a mismatch, check your Git line-ending configuration before suspecting the data.

Validation

A CI workflow checks, for every PR touching data/:

• the committed manifest matches a fresh regeneration;
• source.license appears on the approved list;
• every preview is under 2 MB;
• every .geojson is under 50 MB.