std/pump: Composing Verbs, Not Stores

std/pump is out. Two independently blocking systems — vaxis’s terminal event loop and orisha’s socket server — now compose at compile time into one cooperative driver, share a single kernel poll() over the union of their interests, and drain cleanly together. The contract a participant answers is three verbs and zero branches, and in vaxis’s case the abstraction went all the way down: vaxis:run is no longer a hand-written loop at all — it is a transform that emits the pump itself.

This post is the contract first, the proof second, and what the composition dug out of the compiler last.

What the pump composes

vaxis’s terminal loop (koru-libs/vaxis/index.kz) and orisha’s socket loop (orisha/lib/pump.k) are the actual things std/pump exists to compose. Their contracts, verbatim:

pub tor run { port: u16, tls_port: ?u16, cert_path: ?string, key_path: ?string }
! arrived *Exchange
| stopped string
| failed string

orisha: one effect (! arrived per request, resuming — the loop continues after the arm returns) and two terminals — | stopped, | failed — the only ways out.

vaxis is the same shape scaled up. init answers | ?done and | err string — the only ways the loop ends — and step carries the whole mid-flight surface as forty-odd optional effect arms:

! ?draw koru/vaxis:Window
! ?char koru/vaxis:KeyData
! ?tick i64

Look at what is not there. No | event. No | no-event. No per-iteration answer at all. Inside the body, the loop asks the OS directly — $mod.event_loop.tryEvent() returns ?Event, a value — and “an event happened” is that the optional isn’t null. Everything the loop emits mid-flight is an effect. | is reserved for “the run is over” — done, err, stopped, failed. In both pumps, that invariant holds completely. The contract below is the vocabulary those two loops already speak.

The contract: three verbs, zero branches

A participant is anything that answers up to three verbs — plain calls, compiled per join site into pump-local units:

• step() -> i32 — advance every live instance once; return how many progressed. Required. Progress is a return value.
• live() -> i64 — how many instances are live; 0 = retired. Absent = the participant is immortal, and a pump of immortals runs forever — honest, because that pump is the program’s event loop.
• wait(i) -> { fd: i32, wait_ns: i128 } — instance i’s blocking interest, enumerated 0..live(). Optional; feeds the union wait.

No | anywhere in the contract. The join site is the reference form — the same shape std/store(name) ! field uses for standing watches:

std/pump:create(main)
| drained |> std/io:print.ln("all participants retired")

std/pump(main)
! step |> tick-step()
! live |> tick-live()
! wait i |> tick-wait(i)

std/pump(main)
! step |> tty-step()
! live |> tty-live()
! wait i |> tty-wait(i)

std/pump:run(main)

Pass order is join order — document order. run loops passes until every live answers 0, then fires | drained. A dead pass fires ! idle if armed; unarmed, the pump collects every joined wait interest and blocks once on poll(), capped by the nearest wait_ns.

A store is a participant, not the participant

A ! step/! wait store exposes the three verbs as ordinary generated units — tick-step steps every live row once (retiring | complete in place, counting progress as i32), tick-live is the row count, tick-wait(i) reads row i’s interest. Inside the store, | complete still earns its | — it genuinely retires a row. | stays where it means over, one level down, and the pump never sees it.

And the singleton — the case a store-shaped pump could never express — joins identically. probes/r7 is a module holding raw state (var count, var budget) with three host tors:

std/pump(main)
! step |> lib/single:step()
    ! stepped c |> std/io:print.ln("single stepped {{ c:d }}")
! live |> lib/single:live()
! wait i |> lib/single:wait(i)

It runs, it drains, and the join is indistinguishable from the store’s — which is the whole point. vaxis’s tryEvent()-shaped loop and orisha’s connection set can both sit behind this contract without either one pretending to be a container.

The proof: two real pumps, one driver

orisha/examples/pump-tui joins orisha’s server and a vaxis TUI under one std/pump — the two real pumps from the top of this post, sharing a single poll(). And the composition spells no pump vocabulary at the participants at all: each library’s run is written exactly as it stands alone — the open pump absorbs it. The whole program, verbatim:

// Two natural run calls, one open pump: an interactive vaxis TUI and an
// Orisha HTTP server written exactly as each would stand alone — the open
// `std/pump` absorbs both. One thread, one kernel poll() over the listener
// fd and the terminal's wake pipe, capped by vaxis's tick.
//
//   koruc run examples/pump-tui/main.k
//   curl localhost:3001   — the counter climbs; q quits both.
//
// Nothing in either run call spells pump vocabulary: `create` opens the
// region, `run` drives it, and each library's own run transform emits the
// join units (`__pump_app_{step,live,wait,retired}_<i>`) that `std/pump:run`
// enumerates. `| stopped`/`| done` ride those retire units — they fire when
// the participant's `live()` first answers 0, which is exactly what `q`
// causes for both. The served count is an atomic in lib/stats: `! arrived`
// fires on orisha's worker threads, where a store write would be a race.
//
// A canned answer: the full `raw->answer->reply` pipeline lives in
// orisha/index.k (always loaded via the `orisha` prefix), which drags
// std/compiler into the consumer program — an ambient toolchain leak
// unrelated to the pump composition. The pump verbs stand alone; reply()
// with a static response is enough to prove requests arrive and get
// answered from inside a compound pump.

import std/io
import std/fmt
import std/pump
import orisha/pump
import koru/vaxis
import lib/stats

std/pump:create(app)
| drained |> std/io:print.ln("pump drained")

orisha/pump:run(port: 3001)
! arrived x |> lib/stats:bump() |> orisha/pump:reply(x, head: "HTTP/1.1 200 OK\r\nContent-Type: text/plain\r\nContent-Length: 42\r\nConnection: close\r\n\r\nhello from a pump shared with a vaxis TUI\n", body: "")
    | sent |> _
    | broken _ |> _
| failed e |> std/io:print.ln("orisha failed: {{ e:s }}")
| stopped |> std/io:print.ln("orisha stopped")

koru/vaxis:run(title: "pump-tui")
! draw _ |> lib/stats:count(): n
    |> std/fmt:ln("requests served: {{ n:d }}"): line
    |> koru/vaxis:box(x: 2, y: 1, w: 62, h: 9, border: true): f
    |> koru/vaxis:write-in(win: f, x: 1, y: 0, text: "pump-tui — orisha + vaxis, one std/pump")
    |> koru/vaxis:write-in(win: f, x: 1, y: 2, text: "http   orisha kqueue server — curl localhost:3001")
    |> koru/vaxis:write-in(win: f, x: 1, y: 3, text: "tty    this vaxis dashboard — wake pipe + 16ms tick")
    |> koru/vaxis:write-in(win: f, x: 1, y: 4, text: "wait   one poll() over both interests, nearest deadline")
    |> koru/vaxis:write-in(win: f, x: 1, y: 6, line.text)
    |> koru/vaxis:write-at(x: 2, y: 10, text: "q quits both — retire units tear each down")
    |> koru/vaxis:render()
! tick _ |> _
! char k when k.ch == 'q' |> koru/vaxis:quit() |> orisha/pump:stop()
| err e |> std/io:print.ln("vaxis failed: {{ e:s }}")
| done |> std/io:print.ln("vaxis done")

std/pump:run(app)

orisha/pump:run(port: 3001) is the same spelling a standalone program uses — inside the open pump it emits its machinery plus join units instead of its own driver, and | stopped rides the retire unit std/pump fires on the first live() == 0. Same for koru/vaxis:run and | done. The consumer file carries two pump words — create and run — which is the minimum a bracketed region can carry; everything else is the libraries’ own vocabulary.

Orisha answers GET / while vaxis owns the alternate screen — the boxed dashboard counts requests served, updated off the 16ms tick that arming ! tick puts into vaxis’s wait interest (the only reason ! tick is armed at all; its body is _). The arrived arm bumps an atomic in lib/stats because orisha’s workers fire it on their own threads — a store write there would be a race wearing a demo’s clothes. One q retires vaxis’s live, orisha’s stop follows, both retire units run their deinits, | drained fires, and the process exits 0. The listener fd and vaxis’s wake pipe (the fd-ified stand-in for a condvar queue that cannot be polled) sit in the same kernel wait. No second loop, no thread doing the other system’s work — and no line the consumer would rewrite to run either participant alone.

The part that surprised me: making it true required deleting vaxis’s own driver. vaxis:run was a ~250-line proc hand-spelling exactly the init/step/live/wait/deinit middle the participant verbs already carried — duplicated because the proc could not forward the call’s ! arms. Effect splicing gives a proc __H, the handler struct — usable once, at its own call site, and never as a value. step_event.handler(.{}, __H) is unspellable. The composition a proc cannot express, a generated call site expresses trivially: vaxis:run is now a transform that emits vaxis:step() <your ! arms> as a generated unit — the arms splice there exactly as they do for authored code — plus a generated driver that polls live/wait and calls deinit. And the same transform is what made capture free: it already owned the emission, so “a std/pump is open” became one more branch — emit the same step unit and liveness/wait joins under the pump’s names, move | done onto a retire unit, and let create/run do the driving. The duplicated middle is gone; run literally lowers to the pump — its own, or the open one. A hand-written loop carrying a second copy of dispatch “because the arms cannot be passed on” is a tell: that loop wants to be a generated call.

What it settles

The vocabulary now matches the two real pumps exactly: ! = something happened inside a pass, -> = what a verb found, | = a run is over — and the pump’s only | is | drained, which is exactly that. Composition is compile-time: the joins are sites, the units are generated, the loop is emitted — and now, in vaxis’s case, so is the run that used to fake it by hand. State, plurality, and lifecycle stay inside the participant, where the real pumps already keep them.

One thing it does not settle, and shouldn’t pretend to: the three transforms coordinate by comment markers. create plants // __pump_decl_app as a host line; run and each library’s run transform scan the program’s items for that text. It works — the scans are exact-line and identifier-checked — but it is a string convention duplicated across three files in three repos, and it is already the newest member of a family the codebase has audited and begun retiring: refine moved off this same wire onto a typed program item (facet_decl). The pump markers should follow it — an open pump is a fact of the program, and facts want structure, not grep. That is deferred, not waived: nothing in the consumer surface depends on which mechanism answers “a pump is open,” which is exactly why the move stays cheap.

What the pump dug out of the compiler

The pump’s other job was being heavy enough to break the toolchain honestly — it is a compile-time composition instrument, and it found four bugs in one session:

• A fixed [32] effect-site buffer overflowed once a transform pass held more than 32 sites — allocator-sized now.
• Dead stripping skipped proc bodies for sibling_event.handler references — generated pump drivers lost the very handlers they called.
• A qualified type resolved its home through a bare-name map. ?std.mem.Allocator reduced to Allocator, which a private convenience alias inside [comptime] std.compiler had claimed — so every consumer naming an allocator welded the entire compiler module (AST, passes, logging) into its runtime binary. Qualified spellings now resolve their qualifier; the map is the bare-name fallback. This one is my favorite kind of bug: invisible until a real consumer, catastrophic in exactly the place it should never reach.
• @import("root") references are invisible to reachability — the OpenSSL decls orisha’s generated code calls were being stripped; the public tors carry [retain] now.

None of these are pump bugs. They are compiler bugs the pump was the first instrument wide enough to trip over — the reason the library exists at all is to surface them.