How the pipeline turns PostgreSQL state into the public /v1 JSON contract, what each tier guarantees to readers and how the contract evolves.
| Reference | Value |
|---|---|
| Subject and boundary | Publication tiers, file layout, caching, gates, publication lanes, the historic graph and contract schemas. Edge delivery is on ‣. |
| Verified source | publish.py, storage.py, historic_graph.py, contract.py, live_publication.py, ttl.py at main 9733eb26d92533f56cdc8bc46b81b93fbb5987c8, checked November 10, 2026. |
| Support state | Implemented source. Commands: transit-ops publish-snapshot <provider> --tier ..., publish-all --tier ..., validate-snapshots (findings without upload), verify-historic-publish. |
/v1| Path | Tier | Content |
|---|---|---|
providers.json |
Catalog | Default provider plus each public provider's labels, map bounds, geocode context, poster index, alert links and available inputs; the web app boots from it |
<provider>/manifest.json |
Live | Discovery record for live, static and historic files; rewritten last on every live publish |
<provider>/live/vehicles.json, trips.json, alerts.json, network.json, stop_departures.json |
Live | Current reports, rebuilt only when their inputs change (see Live manifest and changed-only publication) |
<provider>/status/data_health.json |
Live | Per-lane publish freshness and last gate outcome |
<provider>/static/routes_index.json, stops_index.json, routes/<id>.json, stops/<id>.json, basemap.json |
Static | Schedule-derived reference data |
<provider>/labels/<lang>.json |
Static | Localized label dictionaries referenced by the manifest; outside static/ |
<provider>/historic/history/index.json |
Historic | Availability root for seven families: alerts, hotspots, lines, network, receipts, repeat offenders, stops |
<provider>/historic/.../generations/<sha256>/... |
Historic | Immutable, content-addressed children |
| Tier | Write order | Cache-Control | What a reader can rely on |
|---|---|---|---|
| Live | Children, then manifest. Gate findings are recorded, not blocking | public, max-age=20 |
Publication timestamps carry TTL policy; freshness must be checked by the consumer. Files are mutable and can briefly come from different builds |
| Static | Validate, then hash-gate writes; unchanged dataset and fingerprint skip rebuilding | public, max-age=86400, stale-while-revalidate=86400 |
Failures before upload leave the stored files unchanged; a failure during upload can leave mutable files from different publication generations |
| Historic (index) | Build and verify children, compose parents, then conditionally activate the root | public, max-age=3600, stale-while-revalidate=86400 |
The root points at one complete generation |
| Historic (immutable) | Exclusive create by content hash | public, max-age=31536000, immutable |
Bytes at a path never change |
Historic publication refuses known dirty daily history before any storage read, even with --force. Forceable quality findings are separate from mandatory source cleanliness, content integrity and conditional activation.
manifest.files.live field |
Meaning |
|---|---|
ttl_s |
20 seconds from LIVE_TTL_SECONDS, the same value as the live object Cache-Control: public, max-age=20. It is global, not per provider |
source_lag_s |
The provider's declared live_source_lag_seconds: 60 for OC Transpo, 0 for STM. The web app adds it to its live timing |
input_captures |
Accepted capture stamp for each input lane, plus the current static schedule stamp |
generations |
The publish generation of the last completed build of each live file |
live_publication.py maps each file to its inputs:
| File | Rebuilt when this input's capture stamp changes |
|---|---|
live/vehicles.json, live/network.json |
vehicle_positions or trip_updates |
live/trips.json, live/stop_departures.json |
trip_updates |
live/alerts.json |
i3_alerts |
status/data_health.json |
Every publish attempt |
Every file is also rebuilt when it has no stored generation or the static schedule stamp changed. The comparison uses capture timestamps from the prior manifest, not content hashes. Unchanged files keep their previous generation, and a failed upload never advances the stored state.
Payload details:
vehicles.json rows carry next_stop from the vehicle position, or else from the first upcoming stop in the vehicle's own trip update. next_stop_lat and next_stop_lon come from gold.dim_stop rounded to five decimals, only when both exist.vehicles.json still lists vehicles without a trip. network.json excludes them from vehicles_in_service and coverage_pct.Every publisher takes a nonblocking provider and tier lane in PostgreSQL and reads one repeatable snapshot. A competing publisher on the same lane fails before building; other providers and tiers are independent. The realtime cycle records a lane or upload failure and continues capturing.
Remote root activation happens before the publication state is recorded and committed. If the database step fails, the activated graph is visible while database changes roll back; the publisher reports it and does not retry automatically. A retry rechecks the current source and root. The lane cannot fence a remote request in flight after database ownership is lost.
generated_utc follows each tier's own clock: live uses one build-start stamp, static follows dataset loading, and historic uses a UTC-day label. publish_generation_id can therefore repeat across same-day historic corrections; immutable paths and hashes tell those graphs apart.