kaldamus entity reference · EventHistory

Event history

Every full run of update_events appends one probability point per traded event, per UTC day, then compacts the series back down — so a file that used to grow unbounded now has a ceiling by design, not by cleanup.

One row, three blocks

Sub-model	Captured	Holds
HistoryPoint	one per UTC day (key: `date`)	probability at capture · volume_24h · open_interest
HistorySignals	recomputed every advance	delta_1d / delta_7d · volatility_30d · prob_min/max · days_in_band · volume/oi_trend
SettlementSnapshot	written once, on first terminal advance	final_probability (first-write-wins) · resolved_outcome · settled_at · surprise_7d

Row-level fields around the three blocks: id (= event_id) · first_seen · running prob_min/prob_max · points[] · updated_at (derived from as_of, not the wall clock).

Concept

Example

An EventHistory row (id = event_id) is a thin wrapper around three purpose-built sub-models: a raw daily capture, a derived signal block recomputed from it, and a settlement record frozen once.

capture → signals → settlement

No OHLC and no denormalized title/category — events.json is authoritative for those and never deletes, so copying them here would only be drift-prone.

title/category read from events.json, not copied

A same-day rerun is byte-identical because updated_at derives from the capture's as_of, not the wall clock.

rerun same day → identical row

How a row gets built and kept bounded

flowchart TD
    classDef srcNode fill:#16233b,stroke:#5a8fd4,stroke-width:1.5px,color:#dfe7f5;
    classDef signalNode fill:#241d0c,stroke:#c9a84c,stroke-width:1.5px,color:#f3e6bd;
    classDef decisionNode fill:#12151f,stroke:#5a6178,stroke-width:1px,color:#c9cbd6;
    classDef deadEnd fill:#12151a,stroke:#3a4058,stroke-width:1px,color:#7a8296,stroke-dasharray:3 3;

    A["Event row
(full run only)"]:::srcNode --> B{"row exists?"}:::decisionNode
    B -- no --> C{"terminal, or
never traded?"}:::decisionNode
    C -- yes --> Z1["no row created"]:::deadEnd
    C -- no --> D["new EventHistory row"]:::srcNode
    B -- yes --> D
    D --> E{"open AND
probability set?"}:::decisionNode
    E -- yes --> F["capture today's point
(replace if same date)
widen prob_min / prob_max"]:::srcNode
    E -- no --> G["no new point"]:::deadEnd
    F --> H{"status terminal?"}:::decisionNode
    G --> H
    H -- yes --> J["freeze settlement once"]:::signalNode
    H -- no --> M["compact"]:::decisionNode
    J --> M
    M --> N{"terminal AND
30d past last point?"}:::decisionNode
    N -- yes --> O["weekly-only, whole series"]:::signalNode
    N -- no --> P["90d daily window kept;
older → 1 / ISO-week"]:::signalNode
    O --> Q{"over 400 points?"}:::decisionNode
    P --> Q
    Q -- yes --> R["drop oldest to cap"]:::deadEnd
    Q -- no --> S["compute_signals()"]:::signalNode
    R --> S
    S --> T["events_history.json
+ Event.history_signals"]:::signalNode

One call to advance(row, event, as_of, knobs) per event. Blue = raw capture off the live Event row; gold = derived/frozen output; dashed = a path that leaves the row unchanged.

Concept

Example

Stage 3 of update_events composes four pure functions over data already in memory — no model call, no HTTP request — guarded to full runs, exactly like lifecycle reconciliation right before it.

full-run only · zero network · zero LLM

Capture is idempotent per UTC day: a same-day rerun replaces that day's point; a skipped day is just a gap the signal formulas tolerate.

rerun → replace, never duplicate

Bounding is layered, not one rule: 90 days daily, older folded to one point per ISO week, then a hard point cap.

90d daily → weekly → 400-point cap

Terminal events get a second mode — once 30 days past their last point, the whole series collapses to weekly, shrinking toward an archival shape.

settled + 30d → weekly-only

The single upstream is Event itself — never predictions.json directly — preserving the one-upstream-per-stage invariant.

reads Event.probability / signals

How state is managed

Field	Written by	Changes again when…
points (today's entry)	Stage 3 capture, open + traded only	replaced on a same-day rerun; a missed day just leaves a gap
prob_min / prob_max	every capture, before compaction	only ever widen — the cap can drop the point that set an extreme without resetting it
signals	`compute_signals()`	every advance with non-empty points, new point or not
settlement.final_probability	first terminal advance	never again — even if the event later moves closed → settled

Concept

Example

Only a full update_events run touches a row's content, and within it nothing is a blanket "overwrite every run" — each field has its own regime.

scoped --id run → history untouched

The manual candlestick backfill merges Kalshi's own daily closes through the same path — but capture wins on any date collision, so a live point is never overwritten by a backfilled one.

backfill fills only dates the pipeline never reached

History is a bonus lane, not a gate: if Stage 3 throws, it's logged and the run still exits 0 — the completed Stage 1/2 writes are never rolled back.

advance() throws → logged, run exits 0

Transient, or permanent?

Scenario	What happens	Kept?
Open, traded	advances daily; points accumulate then compact	yes — grows toward steady state
Goes terminal	settlement frozen once; captures stop; series rolls off to weekly	yes — this is the calibration record
Orphaned, settled	event gone from events.json, but a settlement exists	yes, forever — exempted from the orphan check
Orphaned, unsettled	event gone, no settlement — no future points possible	no — swept by `--purge-orphan-history`

Concept

Example

Permanent by default — nothing deletes a row, just as events.json never does — with exactly one deliberate exception, added after production evidence.

settled calibration data → kept forever

Reusable settled data was never the problem; unsettled dead weight was — so only settlement-less orphans are swept.

orphaned + unsettled → purged

The purge rides the shared engine — a mass-delete refusal rail and --dry-run — wired into make purge.

refuses to delete >90% of rows

The gap in the data

Concept

Example

"One point per UTC day" is the capture rule, not the cadence — there's no scheduler in the repo, only an operator running make update-all when they remember to.

missed days → real gaps in the series

delta_1d's tolerance is ≤2 days, so a run after a longer gap reports null — an honest absence, not a flat market.

3-day gap → delta_1d = null

delta_7d's longer, no-lower-bound window is far more resilient to the same gaps.

7d window → usually still non-null

A row is only materialized for an event open and traded, so a market that opens and settles between two runs is terminal the first time Stage 3 sees it — and gets no history row at all.

a 15-min crypto strike → price in events.json, no history

Event history

One row, three blocks

How a row gets built and kept bounded

How state is managed

Transient, or permanent?

The gap in the data

See also