std/indexes: a Store Index — Measured Flat, Refusing Silence

A week ago std/store had exactly one way to find a row: sweep. Every ! first p when p.key == key — including the one a game server runs per datagram, per tick — was a linear walk over live rows. Today there’s a declaration:

std/store:new(players, capacity: 1024) { pid: f32, key: i64, /* ... */ }
std/indexes:store(players, key)

That’s the whole user-facing surface. The second line is not a call, not a macro, not a runtime registration — it’s a ~[norun] tor that claims the site and never executes. It sits in program.items through every transform stage. The AST is the registry. The store organ’s verbs find it on their ordinary program walk — the same pattern std/kernel:shape already uses — and each verb emits the maintenance it alone can see:

• new emits the map itself — AutoHashMapUnmanaged(i64, i64) beside the arena, a Map on JS — plus the teardown deinit that keeps the Debug leak checker quiet even on an index-only store.
• insert getOrPuts the new row’s handle.
• take releases the taken row’s key and reseats the next-lowest duplicate — the only place that can see the swap-remove, because no event fires for the displaced row.
• stored re-keys when the indexed column is written.
• query — the payoff — routes ! first p when p.key == k to map.get → handle → dense row. No sweep.

The map stores key → handle, not key → position. Positions move under swap-remove; handles don’t. A row’s generational handle survives relocation (pinned since 690_092), and a removed row’s entry traps loudly through the generational path (690_115/116) — staleness was already solved; the index inherits it free.

Three rulings worth naming, because they were decided rather than defaulted:

Duplicate keys resolve lowest-dense-holder-wins. The map holds the lowest live row per key — the very row the scan’s first match would have found — so routing is invisible to semantics. ! first answers the same row whether it walked or looked up.

Only a bare equality routes. p.key == k hits the map; p.key == k and p.rt > 0 sweeps. Narrow on purpose — a compound guard the index half-answers is a correctness bug wearing a performance costume. Hold that thought; it becomes the punchline.

The namespace is std/indexes, plural. koru_std/index.kz is the std root module — std/index as a qualifier is structurally impossible (the resolver refuses index.kz + index/ coexisting, KORU200). The concept file recorded std/index in the design phase and Lars corrected it at ship time. The docs drifted; the commit dragged them to the truth.

Then we pointed it at a real server

games/asteroids-net is the ecosystem’s forcing-function slice — the Server Challenge replay: an authoritative UDP server, a bound socket, a 60Hz tick loop, the forked asteroids sim, one snapshot per client per tick. Its entire input path is one tor:

pub tor drive { key: i64, tick: f32, seq: f32, left: bool, right: bool, thrusting: bool, firing: bool }

drive = std/store:query(players)
    ! first p when p.key == key
        |> if (seq > p.lseq)
            | then |> std/store:stored { p.last: tick, /* ... */ }
                |> control(p.pid, left, right, thrusting)
                |> fire(p.pid, pressed: firing)
            | else |> std/store:stored { p.last: tick }
    | none |> /* find-or-join: spawn-player, spawn-ship */

Per packet: find the session row by its UDP key fold, apply input, or join if unknown. Exactly the shape the index was built for. The change was two lines — the import and the declaration — and the emitted Zig shows koru_sim.__koru_store_players.__index_key.get(key) where the sweep stood.

Then we measured. load.sh 64 — 64 real client processes streaming sequenced input at ~60Hz, real datagrams, 600 ticks, server heartbeat decomposing the tick budget per phase:

Flat. Everything drifted up together — that’s run noise, not a regression. The index routed correctly and the number didn’t move.

Why flat is the finding

Look at what drive actually does per packet. After the session lookup — a ≤64-row scan — it calls control and fire, and each of those runs ! first over the bodies store:

! first e when e.kind == 1.0 and e.owner == pid and e.dead == 0.0

A compound predicate over hundreds of entity rows — ships, rocks, live bullets — run twice per packet. The session-row scan the index eliminated was the small term. The honest arithmetic: 64 i64 compares is sub-microsecond work; the two bodies scans are where drive’s ~6.7µs/packet actually goes.

And those scans can’t route — today. They carry and-joined conjuncts, and the shipped ruling is bare-equality-only. Which names the next rung with unusual precision: conjunct routing. Extract e.owner == pid from the and-chain, route it to a bodies.owner index, evaluate the residual kind/dead conjuncts on the single resolved row. Same semantics — the resolved row is re-checked against the full guard — and both bodies sweeps become map lookups. That’s where the drive number lives; that’s the wall the instrument just pointed at.

This is what “oracle-backed” is supposed to feel like. The alternative history is easy to imagine: ship the index, declare victory in the commit message, never run the load probe. The flat line is the receipt that the measurement is real — a feature whose first honest benchmark doesn’t move is not a disappointment, it’s the toolchain telling you the truth about where your server’s time goes. The index was never the product. The instrument is.

The registry’s silent shapes

But a registry whose answers are only honored or invisible has a failure mode worse than an error: a claim the author believes is live, and the machine dropped it without a word. Two shapes of that drop shipped with the feature.

A second claim on one store. std/indexes:store(players, key) followed by std/indexes:store(players, val) compiled clean — the discovery walk returned the first match and nothing else looked. The author believes val is indexed; every ! first on it scans forever, and nothing anywhere says so.

A claim naming no column. std/indexes:store(players) read the same as no declaration — the walk found the node, found no column on it, and moved on. Dead text that looks live: no field, no machinery, no diagnostic.

Both are refusals now, under one code, at the claim’s own location (line-wrapped here — the compiler emits each on one line):

error[KORU161]: std/indexes:store(players, val): 'players' already carries
an index on 'key' - a second index on one store is a later rung
  --> input.k:17:0

error[KORU161]: std/indexes:store(players): the declaration names no column -
an index keys on one column of the store: std/indexes:store(players, <col>)
  --> input.k:13:0

The diagnostic points at the declaration, not the store — the claim is the thing that’s wrong — and each message carries its teaching half: what the rule is (one index per store; an index keys on one column) and what the correct spelling looks like.

The mechanism sits where the ruling sits — inside the organ, not beside it. A new ~part, store.idecls.kz, holds StoreIndexDecls: declsOf collects every declaration naming a store in one walk, and fieldSpec owns the whole decision the new emit makes — the backing map field and the keyed column, or a refusal. columnOf, the read the query router uses, is now a first-match over the same list. One collecting walk, two consumers, and no way for them to drift apart.

This is the organ completing its own discipline rather than a new rule bolted on. The index already refused a claim on a scalar-typed column and a claim naming a field the store doesn’t have; a declaration the machine can’t honor is the same species of claim. The line the registry now holds: a declaration is either live or loud — never dead text. That isn’t tidiness. A silently-dropped index is a performance belief the compiler signed and never delivered; the program it would have saved scans on, unaware.

What the index says next

The refusal is the honesty half of the story. The other half is the index saying its plan — an explainer row in the catalog koruc explain already gathers, answering the question the flat measurement made you ask by hand: did my query route through the map, and which index served it? That surface is in flight, and the refusal texts are already shared constants, so the diagnostic and its explanation cannot drift. Further out is the rung the flat line named: conjunct routing, splitting e.owner == pid out of compound guards so the map answers the scans that actually cost.

The organ’s arc in one pass: declared, routed, measured honestly, taught to refuse — and about to start explaining itself. The server it measured against, and the ladder it climbed, is the Server Challenge.

Commits: koru a6de55946 (the feature), e629e2134 (the | none body-builder split that bought store.query.kz headroom), 815689527 (the red pin), 084904cdd and 220bb8a52 (the refusals). koru-libs 39051e0 (the consumer). The consumer: games/asteroids-net/sim/index.k.