What changed in the archive, the bundles and the delivery paths — newest first. Entries marked FORMAT change the bytes of a file or the shape of an API response: those are the ones worth reading before your next pull.
Capture itself is continuous and is not a release — the archive grows every day without an entry here. This page covers changes you could notice from the outside.
perps_instruments no longer stores a symbol column
In the perps bundles every table is filed under a symbol= partition, and the instruments dictionary also stored symbol as a column, so reading that one directory as a tree failed with "Field symbol has incompatible types". The column is gone: symbol now comes from the partition path in all six tables, exactly as it already did for the five event tables, and the dictionary reads the same way as the others. Existing days were rewritten in place; nothing else about the files changed. This landed hours after the venue itself, so only bundles pulled on 2 September before the change carry the old shape.
what to do
Read perps_instruments like any other table (read_parquet on its directory). If you pulled a perps bundle earlier on 2 September and worked around the collision with partitioning=None, that read now returns no symbol at all — drop the workaround.
Polymarket perps: order books, BBO, trades and funding for the perpetual futures venue
Polymarket's perpetual futures venue is now a second venue in the catalog, captured on two independent collectors since 2026-07-31 and normalized nightly into six tables per UTC day: perps_bbo (event-driven top of book), perps_book (20-level snapshots every ~100 ms), perps_book_deep (the full ladder every ~15 s from REST), perps_trades, perps_tickers (mark, index, last, open interest and the funding rate) and the perps_instruments dictionary. 67 instruments across crypto, US equities, indices and commodities. It is sold per day with an instrument filter, included with Premium and available as a monthly add-on on Explorer; every account can pull one fixed sample day. Bundles keep the Hive layout (dataset/date=/symbol=/) so one read_parquet reads a whole event table; the dictionary stores symbol as a column as well, so read it with partitioning=None. Books are full snapshots rather than deltas, funding lives only in the ticker table, the first day is partial (capture began 15:44 UTC) and KPEPE-USD is absent on 2026-08-04 and 2026-08-05; the README states these for your exact range.
what to do
New endpoints: GET /api/v1/perps for coverage and access, POST /api/v1/perps/downloads for a pull; poll GET /api/v1/downloads/{id} as usual. Schema reference at /docs/perps. Perps bundles have no CSV twin.
Order-book history now reaches back to 21 February 2026
Market·day coverage now starts on 2026-02-21 rather than 2026-05-11, adding roughly eleven weeks in front of our own capture across 5,277 markets. The extension is licensed third-party archive data converted into exactly the layout you already parse — same partition paths, same column names, same join keys — so nothing about how you load a bundle changes. What differs is worth knowing before you join across the boundary: trade tape on the extended window begins on 2026-04-13, so a market·day between 2026-02-21 and 2026-04-12 ships l1.parquet and l2.parquet with no trades.parquet, and the bundle README says so rather than shipping an empty file that would read as "no trading happened". On those earlier dates ts_recv_ns carries millisecond resolution (its last six digits are always zero) and msg_seq is a generated per-day counter rather than the venue's sequence number, so don't build cross-boundary gap detection on it. Raw websocket atoms (Enterprise) are unchanged and still begin 2026-05-11 — original frames for the earlier window do not exist and cannot be produced. On-chain settled fills are a separate layer and continue to reach back to 2023, per market.
what to do
If you pull a fixed date range, you can now request earlier days than before — check the per-unit range on the market's page or via /api/v1 rather than assuming a single floor, since coverage starts when each market first traded. If your loader requires trades.parquet to be present, guard for its absence on dates before 2026-04-13.
Candles: interactive viewer, and 1m candle CSVs in bundles
OHLCV candles for every market on the venue are live at /candles — built from our own capture, two independent sources with one clock each: the trade tape (OHLC, VWAP, volume, taker-side split) and the order-book midpoint (OHLC, closing bid/ask, time-weighted spread; one-sided, crossed and placeholder books excluded). 15-minute and coarser charts are free for everyone, full history; any paid plan unlocks the intraday band down to 5-second bars, a trades+midpoint overlay, and — with feeds access — a crypto spot pane under the market. Bundles can now carry the candles too: POST /api/v1/downloads accepts candles: true/false and ships candles/tape_1m.csv + candles/mid_1m.csv inside both twins (small zipped CSVs, deliberately approachable next to the tick parquets). Free market·day downloads include them by default, and the free sample bundle now carries them as well.
what to do
Existing accounts keep receiving exactly the bundle members they always did — candles land only when you ask (the checkbox on /catalog, or candles: true on the API). Accounts created from today include them by default; pass candles: false to opt out. When present they add a candles/ directory; parsers that walk the zip by fixed member list should key on names.
Every fill as settled on Polygon — price, size, taker side, and the taker wallet, which the websocket tape has never carried — is now downloadable per market and date range from Premium upward. It is a separate pull rather than a file inside the market·day bundle, because its coverage is different: a market's chain history begins when that market first traded, which is usually months before our book capture, and thousands of markets closed before capture started and have chain fills as their only record. Request a range with POST /api/v1/onchain (31 days max, poll it through the same GET /api/v1/downloads/{id}), or browse coverage with GET /api/v1/onchain, which publishes each unit's exact first and last chain day. Timestamps are second-resolution block times, so use the tape when you need sub-second ordering; both sources carry transaction_hash and join exactly where they overlap.
what to do
Coverage is per market and we publish it per market — read first_date/last_date from GET /api/v1/onchain rather than assuming a common floor. A date before a market traded has no fills anywhere, which is a launch boundary and not a gap.
Idempotent backfill clients kept guessing list endpoints and re-requesting days they already had, so the collection URL now answers GET: your jobs, newest first, with state, byte count, the poll URL and — once ready — the download URL. limit= up to 200, state= to filter. A ready job's URL stays fetchable for the whole 30-day access window, so the efficient loop is list, reuse, and only POST what is missing.
Feed files align with the market format — ts_recv_iso and payload_sha256 removed
The live-feed datasets (rtds_prices, sports_state) no longer ship ts_recv_iso or payload_sha256 — the same trim the market-day files got on Aug 9, which the entries below said feeds were exempt from; that exemption ends today. ts_recv_ns remains the receive timestamp, every other column is unchanged, and rtds day files are about 64% smaller for it. Historical feed day files were rewritten the same day with identical rows, and the spot ride-along files inside market bundles drop ts_recv_iso as well.
what to do
Do not rely on ts_recv_iso being present anywhere in normalized output — it is stripped for efficiency and will not be provided on an ongoing basis. Files pulled before 2026-08-18 (and bundle spot files built before then) may still carry the two columns; select columns by name, and rebuild the ISO form when needed: pd.to_datetime(df.ts_recv_ns, utc=True).
Interrupted downloads resume instead of restarting
Bundle downloads — dashboard, REST API and SFTP — now serve HTTP range requests properly, so a transfer that dies at 80% picks up where it stopped rather than starting the whole file again. curl -C -, wget -c and resuming SFTP clients all work against a multi-gigabyte archive pull now, and our SFTP server retries a broken upstream read mid-stream instead of failing the transfer.
When a bundle's market and date overlap a known capture gap or a Polymarket maintenance window, it now ships a known_gaps.csv listing each affected window with its severity, cause and a note, and the README gains a Known gaps section. Clean days ship without the file, exactly as before. Feed pulls of rtds_prices carry the same file when their range overlaps a gap.
what to do
If you scan bundles programmatically, treat known_gaps.csv as optional — its presence is the signal, its absence means we know of nothing wrong with that day.
Roaming plans pull continuously, and a busy archive crawl was generating thousands of "your data is ready" emails that only repeated what the dashboard already showed — enough volume for Gmail to start deferring our mail. Only discrete things you are actually waiting on are announced by email now: à-la-carte orders and free claims. Everything else is visible in the dashboard and over the API the moment it is ready.
Setting up SFTP no longer needs us in the loop. Generate a key pair in your account (the private half is created in your browser and never reaches us) or register one you already hold, pick your markets, choose parquet or CSV, and your account page tells you when the mounts are live. You land in two folders: /requests, every bundle you hold grouped by date, and /ongoing, a rolling window of the live normalized tree scoped to the markets you selected.
Both the dashboard and POST /downloads now take the bundle shape you actually want: full depth (the default, unchanged) or top-of-book only — L1 plus trades, with no L2 bytes. A top-of-book bundle of a busy series is a fraction of the size and builds in a fraction of the time. Free claims cost one slot either way.
what to do
Pass layers on POST /downloads to skip L2. Omit it and nothing changes.
ts_recv_iso removed from l1/l2/trades — bundles are ~16% smaller
Every normalized row carried both ts_recv_ns (nanoseconds since the Unix epoch) and ts_recv_iso, an ISO-8601 string spelling of the same instant. The string is derivable from the integer and cost 16.4% of the normalized footprint for zero information, so we stopped writing it. Measured on one real series-day: l1 28.5% smaller, l2 15.2%, trades 7.0%. Every other column is bit-identical. ts_recv_iso is unchanged in raw.jsonl (it is part of the capture envelope) and in the live-feed datasets rtds_prices and sports_state.
what to do
If your loader reads ts_recv_iso, rebuild it: pd.to_datetime(df.ts_recv_ns, utc=True). Select columns by name rather than by position or count — the free sample bundle is a pre-change vintage and still has the column, and some historical l2.parquet files still carry it while the backfill finishes, so both widths are in circulation.
payload_sha256 no longer shipped in the raw archive
The raw capture envelope carried payload_sha256, our own hash over payload_text. It is spent once deduplication has used it, and it was about a third of the raw archive's bytes, so it is no longer delivered. Nothing about how we deduplicate changed — the hash is still computed at capture — and it stays reproducible from the file you hold as sha256(payload_text). The feed datasets are unaffected and still ship the column.
Live-feed datasets: crypto spot prices and sports game state
Two feeds we were already capturing are now available as whole-day catalog items: rtds_prices (Binance spot trades and Chainlink oracle prices relayed by Polymarket's real-time stream) and sports_state (live game states — score, period, status). They are included on Premium and Enterprise and available as an add-on below that, and they pull over the API through POST /feeds/downloads. Neither carries a condition_id — join them to market data on time.
A quickstart notebook you can run before signing up
The quickstart walks a real bundle end to end in pandas — load, join to the reference tables, rebuild the book, measure a spread — and it is ungated: readable in the browser, downloadable as an .ipynb, and pointed at sample files anyone can fetch. No account needed to check whether the data is shaped the way your research needs.
The catalog got the filters it was missing: by vertical (sports, politics, crypto and the rest), by kind (series or standalone event), by whether a unit covers a specific date, and by whether its L2 ladder is already built — plus sorting. Finding the ten market-days you actually want in a venue-sized catalog is now a couple of clicks rather than paging.
Bundles build about a third faster — and tell you how long they'll take
The staging pipeline was compressing the same bytes three times over; it now encodes once and reads from the fastest copy of the source data available to the worker, cutting roughly a third off the wall-clock of a typical build. The dashboard shows a live estimate for a job in flight, measured from the work actually remaining rather than guessed from an average, and a pull that has died now says so instead of looking slow forever.
Repeat pulls are instant, and delivered bundles stay reachable for 30 days
Bundles are now addressed by what they contain rather than by who asked for them, so a market-day someone already built is handed to you immediately instead of being rebuilt from scratch — which for a busy series was over an hour of L2 reconstruction. Two clocks govern it: the shared object lives seven days, and your access to anything delivered to you lasts 30. A repaired day misses the cache by design, so you never get stale bytes after we fix something upstream.
A live SFTP endpoint at sftp.tickfoundry.com:2022, key-authenticated, for customers who would rather mount the archive than click a download button. Included on Premium and above.
The product moved off its prelaunch page onto tickfoundry.com: self-serve accounts, checkout, the catalog of every market-day on disk, dashboard delivery, and the first research note — an anatomy of one match's order book, computed from the same bundles we sell. Served over HTTPS with HSTS from day one.
Bundles now include a reference/ directory — markets, events and tokens as both CSV and JSON — covering exactly the identifiers present in that bundle. A condition_id or token_id in l1.parquet maps to the real question text and outcome labels without a round-trip to the API or the venue.
The read API shipped at /api/v1, available on Explorer and above. Listing the catalog, requesting any market-day, polling the build and streaming the bundle down are all scriptable with an API key — no dashboard round-trip, and the same delivery path the dashboard itself uses.
Older than this
Everything before June 2026 predates the public product. Tick and book capture has run continuously since 2026-05-11 — the coverage story lives in the data-quality section of the docs rather than here.
Something changed under you that isn't listed? Tell us — an unannounced format change is a bug on our side.