Running the pipeline
Source YAML in, PMTiles out — how OpenHunt turns government GIS downloads into map layers.
The pipeline is a Rust CLI that turns entries in the source registry
(sources/) into PMTiles archives. Rust owns registry parsing, caching,
sequencing, and validation; the heavy geo lifting is delegated to
battle-tested tools — GDAL's ogr2ogr for normalizing, tippecanoe for
tiling.
Every run is idempotent and resumable: downloads are cached, outputs are written temp-then-rename, and invariant guards fail loudly rather than publish a half-built layer.
Prerequisites
- Rust (to build the CLI)
- GDAL (
ogr2ogron PATH) - tippecanoe on PATH
cargo build --release -p openhunt-pipelineRunning a source
One source YAML → one PMTiles archive:
./target/release/openhunt-pipeline run sources/us/padus-co.yamlOr everything in the registry:
for s in sources/us/*.yaml sources/us/*/*.yaml; do
./target/release/openhunt-pipeline run "$s"
doneOutputs land in data/out/<layer>/. For a self-hosted web app, copy them
into the tiles directory the web tier serves and restart the web container.
The stages
Download. Fetches the source's URL into a content-addressed cache — archives are unpacked, ArcGIS feature services are paged through automatically. A re-run with a warm cache skips straight to normalize.
Normalize. ogr2ogr reprojects to a common CRS and applies the
source's SQL field mapping, producing NDJSON in the common schema for its
layer class (parcel, hunt_unit, public_land, …).
Tile. tippecanoe builds the PMTiles archive with the zoom range and
flags recorded in the YAML.
Derived layers
Some layers aren't downloads — they're computed:
landlocked— public land with no lawful route in, from PAD-US + OSM roads.terrain— slope and aspect rasters from USGS 3DEP DEMs.unit-profile— per-hunt-unit stats (percent public, landlocked acres, road density, public land more than a mile from a road) that power the web app's unit pages.
Where this is going
Published artifacts will be versioned and promoted by manifest pointer flip, so a bad pipeline run can never poison serving and rollback is instant. The registry stays the community contribution surface — see adding a state.