Concepts
Data layers
Recorded tape stays in object storage. Each product surface has a payload sized for that job — the website never scans hours of L2 objects.
Three paths
| Layer | Job | What moves |
|---|---|---|
| Hot preview | Market page: recorded window, YES price, snapshots, and L2 market data | Cached recorded range, ≤40 YES-path points when the book moved, up to 8 snapshot rows, and up to 8 L2 rows. |
| Warm snapshots | Interactive research | A bounded window, max 6 hours, downsampled with step_ms. |
| Cold export | Backtests and pipelines | Named files: snapshots (sampled top-of-book) or L2 (book arrays). Bytes stay on S3 until you download. |
Preview cache
Repeat views of the same ticker are one GET of state/preview/{ticker}.json.gz (plus an in-process hit). A miss returns pending: true immediately and rebuilds off the request thread. Live markets refresh about every 20 seconds. Closed markets stay cached for a day. The lake is not rebuilt on the page-load request.
Choose the right call
GET /api/v1/markets/{ticker}/preview # coverage + path + snapshots + L2
GET /api/v1/markets/{ticker}/snapshots # research window (max 6h)
POST /api/v1/exports # snapshots or L2 file