A map with no legend is just ink.
Legend reconstructs the history of United States energy distribution networks — natural gas, oil, and electric transmission — as a temporal GIS dataset, from the archival record.
The end product is a timeline: a map you can scrub through, decade by decade, watching the country wire and pipe itself up. That artifact is intended as an input to both research and policy discussion. This repository holds the machinery that builds it.
A georeferenced historical map gives you lines. It does not tell you what the lines were, who built them, when, or what they carried — the legend is missing, and for most of these sources it was never written down anywhere in the first place. Legend reconstructs it.
Historical energy infrastructure survives mostly as maps — pipeline and transmission routes overlaid on political base maps, printed in trade journals, annual reports, permit filings, and atlases. Georeferencing those maps recovers where a line ran.
It recovers nothing else. The attributes that make the data useful are not on the map:
| Attribute | Where it actually lives |
|---|---|
| Year built / in service | trade press, permit filings, annual reports, PUC dockets |
| Owner and operator (and their changes over time) | corporate filings, FPC/FERC records, news |
| Role in the network (trunk, gathering, spur, intertie) | engineering write-ups, system maps |
| Diameter, pressure, voltage, capacity | journals, regulatory submissions |
| Fields, plants, or load centers connected | narrative sources |
| Retirement, conversion, rerating | later editions of the same sources |
So the geometry comes from one kind of source and everything else comes from a scattered, contradictory, largely un-digitized textual record. Recovering the second half is the actual work, and it is why this is an AI project rather than a digitization project.
Three components, in dependency order:
1. A data profile (spec/) — a documented, versioned schema for dated, sourced
network edges and nodes. The core is small: a GeoJSON feature, a carrier, an existence
interval that may be imprecise, and a citation. Optional extensions handle sources that
disagree and attributes that changed. Start with
spec/quickstart.md.
2. An agentic extraction system — takes a georeferenced map (or other input) and produces candidate nodes and edges, then goes after the supporting metadata across the open record, with weighted attention to source classes known to be productive: oil and gas trade journals, permit and certificate applications, regulatory dockets, newspaper archives, annual reports.
3. A correction loop — researchers review, label, and correct output; corrections feed back as supervision. A correction is an ordinary edit with a commit message naming the source that changed the reviewer's mind. The repository history is what makes the corpus auditable, so nothing has to be retained inside the records.
- Not a pipeline modelling or flow-simulation tool. It records what existed and when.
- Not a general historical GIS platform. Scope is US energy distribution networks.
- Not the dataset. The published corpus and the timeline visualization will live in a
separate repository (
us-energy-timeline) so that it can be versioned, DOI-minted, and cited independently of the code that produced it.
These are the decisions that shape everything else. Each is argued in
docs/decisions.md.
The data is GeoJSON, and releases are GeoPackage. Requirement zero is that the output
opens in QGIS and the rest of the field's tooling without conversion or plugins. GeoPackage
carries true date columns, which GeoJSON cannot, and that unlocks QGIS's native Temporal
Controller — an animated map of the network growing, with nothing installed. See
spec/profile.md §5.
The simple case stays simple. A conformant record is a GeoJSON feature with a carrier, an existence date, and a source string. That is the expected shape of most Legend data permanently, not a starter subset to be outgrown. Source disagreement, attributes that changed over time, ranks and provenance graphs are optional extensions for the cases that warrant them — which is a minority of features.
Dates are imprecise and the schema says so. "Circa 1928", "sometime in the 1930s", "before 1941" are the normal case. EDTF encodes them honestly; an integer year would launder uncertainty into false precision.
The repository history is the audit trail. No revision or assertion-time fields in the records. Git already records who changed what, when, and what it replaced, with better integrity properties and no authoring burden — so a correction is just an edit with a good commit message. This is what makes releases reproducible and citable.
Provenance is per claim where it matters. A single edge may take its geometry from a 1931 atlas and its operator from a 1934 filing. Citations can be a plain string when that's all there is, or a resolvable identifier — for scanned maps, an IIIF Georeference annotation URI, which lets a reader check traced geometry against the warped source.
License lanes stay separate from day one. OSM-derived geometry (ODbL, share-alike), Global Energy Monitor data (CC BY 4.0), and US federal data (public domain) do not compose cleanly. Provenance is tracked per geometry so publishable subsets can be extracted later. Retrofitting this is not possible.
spec/
quickstart.md the whole format for the common case — start here
profile.md the full profile; conformance levels, extensions, distribution
docs/
architecture.md extraction pipeline and correction loop
sources.md source survey, per-source license lane and expected yield
labeling.md guide for researchers reviewing and correcting output
example-louisiana-1930.md one real sheet, end to end — the reference case
decisions.md decision records: why these choices and not others
Where to start: researchers and data consumers want
spec/quickstart.md,
docs/example-louisiana-1930.md, and
docs/labeling.md — nothing else. The rest is for implementers.
Pre-alpha. The profile is the live question; nothing downstream is stable until it settles. The profile is what the research group needs to agree to first, so it is the first thing written and the first thing to review.
Recorded here because each one has already cost somebody a dataset:
- OSM full-history is edit history, not asset history. A changeset timestamp says when a mapper drew a line, not when it was energized. These must never be conflated in training data.
- OSM retires infrastructure by deletion, not by adding an end date, so it systematically under-represents the past.
- Entity resolution is the hardest modelling call here. A line rebuilt at higher voltage, or a gas line converted to oil, is either the same asset or a successor. Decide early, record merge and split lineage, and never let the answer be implicit.
- No open US historical transmission geometry with build dates exists. HIFLD has geometry and no dates; EIA-860 has real in-service and retirement years but points only. Joining them is original work, not a data-loading step.
Code and the profile specification are MIT; see LICENSE.
The dataset's license is a separate, still-open question, and MIT is not the answer to
it. MIT is a software license and says nothing useful about a database. The choice is also
not entirely ours to make: what data/ can be released under is constrained by the license
lanes of the sources it ingests, and ODbL share-alike in particular propagates into
anything derived from OSM geometry. See docs/decisions.md 0006 and
0009. Settle it before the first release, not after.