Kopium: The Partial Program — a session journal

Today was a long bench session on kopium — the agent harness whose only tool is a Koru interpreter. This is a journal of what the day actually produced, which turned out to be less about a scoreboard than about a failure shape the wire makes visible: the partial program — a reply that runs the prefix of itself and is refused the tail.

The wire, in one paragraph for the reader arriving cold: the model’s reply is a program, not a message. verb(arg: "...") | ok name |> calls and binds; {{ name.field }} inside a quoted string splices a bound value into the prose the user hears; steps run in order and each reports back step k/n result: ... bound: rN. Every verb declares its outcomes up front — | ok string | error string on the tools here — and an outcome that fires unarmed stops the reply at that step with unhandled-branch. One system message carries the whole contract — grammar, vocabulary, generated tool docs, domain policy. Everything else in the context is just the conversation.

The register block is the whole contract

Worth being concrete about what “the vocabulary is a declaration” means, because this part has survived every refactor. An airline agent is a register block and fourteen tors — this is the real generated surface, trimmed to one verb:

[
- doc
Get the details of a user, including their reservations.
]
pub tor get-user-details { user_id: string }
| ok string
| error string

std/runtime:register(scope: "airline") {
    say(1)
    book-reservation(10)
    calculate(1)
    cancel-reservation(10)
    get-user-details(1)
    // … nine more …
    update-reservation-passengers(10)
}

Reads cost 1, writes cost 10 — the budget weight of a commitment versus a lookup is declared, not improvised at dispatch. From that one declaration the runtime derives every surface the model ever meets:

• The grammar it may emit — invocations over exactly these verbs, enforced per step by the scanner, not suggested by a prompt.
• The vocabulary render — the pub tor lines plus their - doc annotation text land in the system message verbatim (”Get the details of a user, including their reservations.” is the annotation, machine-read, not prompt-writing). The documentation the model reads and the declaration the dispatcher enforces are the same bytes.
• The outcome law — | ok string | error string names the legal arm words; an invented arm refuses before the call runs, and today’s ruling lives here: a failure fires its own branch or stops the reply.
• The contract doc — argument shapes, enums, and nested field shapes generated from the same schema the stock arm uses, so the wire arm’s contract is the stock arm’s schema by construction.
• Growth — a tor + name = chain definition installs a verb into the session’s own environment, rendered back into the vocabulary on the next turn.

The scope parameter is the whole seam: swap airline for the notes scope and the same runtime, bridge, and grammar enforce a different world — including the possession annotations (handle: string<!open> on an arg, -> string<open!> on a mint), which is how the native airline surface refuses a payment_id the user never had without asking tau2 at all.

The sandbox is a scope; the loop is one call

The host side is smaller than the declaration. A bridge is opened on the scope — that handle is the sandbox, the only door between the model’s text and the world — and grammar renders the vocabulary it will enforce straight into the system message:

[with]std/bridge:create(id: "kopium", scope: "airline",
                        world: "runs/task32"): br
|> std/bridge:grammar(br)
| ok sys |> // sys is the system message — the bytes the
            // model reads are the bytes the scanner enforces

Then every reply is one run against that bridge — the model’s text goes in as source, the world state persists in the named world, and the outcome comes back tiered. This is the turn tor from the regression suite’s sandbox agent (440_041), where model(n) is a scripted proc standing in for the API call — in the bench harness the same line is the ask:

turn = model(n): wire
|> std/bridge:run(br, source: wire)
| result r            |> // value, bound names, per-step trace
| defined d           |> // a tor just installed — re-render the grammar
| unhandled-branch u  |> // a branch fired that nobody armed
| error e             |> // refusal, lossless — e.kind names the class

Four arms is the whole read side: findings you may arm (defined, unhandled-branch, partial, residue), failure you must (error — e.kind decodes back to the typed class when a caller wants it). The model never sees this tor — it sees the grammar; the harness sees this — and that is the entire tool channel.

Chains nobody trained it on

GLM-5.3-flash — a mid-weight model nowhere near the tau2 board’s usual frontier entries — read the grammar block and started composing. Its second turn on task 39, verbatim from the receipt:

get-reservation-details(reservation_id: "8C8K4E") | ok c1 |>
get-reservation-details(reservation_id: "UDMOP1") | ok c2 |>
get-reservation-details(reservation_id: "XAZ3C0") | ok c3 |>
get-reservation-details(reservation_id: "LU15PA") | ok c4 |>
get-reservation-details(reservation_id: "MSJ4OA") | ok c5 |>
get-reservation-details(reservation_id: "I6M8JQ") | ok c6 |>
get-reservation-details(reservation_id: "4XGCCM") | ok c7 |>
say(text: "Magnifique — pulling up all seven of your reservations now, one moment please...")

Seven fetches, seven bindings, one line to the user — dispatched step 1/8 through 8/8, every result reported back bound. The reply after it was a single say that walked all seven reservations against the cancellation policy in one message. The fetches were the program; the say was the epilogue. This is the texture the wire was built for: the model decides how much work one reply carries, the interpreter dispatches and accounts for every step.

The turn that ran halfway

The more instructive receipt is the one that broke. GLM’s very first wire turn on task 32:

get-user-details(user_id: "michael_smith_1234") | ok u |>
say(text: "Sorry to hear about your mother, {{ u.name }}. I need your user ID
to locate the reservation — could you provide it?")

The guessed ID doesn’t exist — and this receipt is from before the fix, which is exactly why it earned a place here. In this era the harness flattened tau2’s structured error flag into the result string ("Error: User michael_smith_1234 not found"), so the ok arm fired anyway, and u bound an error payload rather than a user record. When the tail tried to splice {{ u.name }}, dispatch refused, and the report the model saw named exactly what broke:

step 1/2 result: {"branch":"","value":"Error: User michael_smith_1234 not found"} bound: r1
step 2/2 dispatch-error say: '{ u.name }' — the bound value 'u' carries no field 'name'

That is a partial chain — partial is a counted verdict class in every session’s ledger, distinct from residue and unhandled. Step one ran and its result is durable (bound: r1); the say never dispatched; nothing reached the user. What saved the turn was incidental: the error text carried no name field, so the splice refused. Had the payload contained the key, the lie would have shipped — the semantics underneath the honest report were wrong.

That receipt is why every tool on the surface now declares two outcomes — | ok string | error string — and the environment’s error flag picks the branch. The same program today reports at step one unhandled-branch: error — {"branch":"error","value":"Error: User michael_smith_1234 not found"} (held []), teaches the recovery spelling | error e |>, and no say ever leaves — later steps that leaned on the never-bound u report as refusals, not silence. Arm the error and it binds as e — a reason the model can act on, not a record it mistakes for a user. The interpreter’s job is unchanged: evaluate the program honestly to the point where it stops being honest. What changed is that failure now arrives on a declared branch instead of hiding inside the one that fired.

A caveat worth its own sentence, because the ledger records it too: chains deliver whatever they bind, even when what’s bound is ugly. On task 29 the model spliced two empty searches straight to the user’s ear — Outbound options 5/17: [] ||| Return options 5/19: [] — every step dispatched, nothing refused, raw brackets delivered. The wire guarantees the call’s shape, not the quality of its prose.

What the day measured

Some housekeeping numbers first, because one of them mattered: mid-session we found that tau2’s user simulator stamps the agent’s usage onto its own UserMessage objects — so a transcript filtered by “messages carrying usage” had been summing the simulator’s calls into the agent’s column. Every earlier A/B number was quietly inflated. With the count rebuilt to agent messages only, the honest matched pairs (same model, both arms, both tasks won):

task 23   koru 177,820 total tokens   stock 295,464
task 31   koru  39,431 total tokens   stock  52,818

The wire surface carries ~40% less context on the long task and ~25% on the short one — the system message is byte-stable across turns, so over 80% of GLM’s prompt bytes came back as cache reads on the provider side. These are two matched tasks and single draws, not a benchmark: the token accounting is honest now, but the sample is small and the numbers are provisional — the work in progress is more draws, more tasks, and more models before the claim hardens.

Two more accounting fixes changed the measurements themselves. The user simulator now runs at temperature 0 — tau2’s own default, which our harness wasn’t passing — and with the sim pinned, GLM went 3-for-3 on task 32 (22, 20, 22 calls) where unpinned draws had gone 1-of-2. Draw variance was partly the benchmark’s randomness, not the agent’s. And a 50-task koru-only sweep under muse landed 47/50 (94%) — with the losses being the interesting part.

Three losses, three different things

• Task 29 failed under every model and draw we tried (five attempts across three models): a multi-leg rebooking that grinds through one-stop flight analysis and never completes. Task-hard.
• Task 32 failed under muse twice and deepseek once — and passed under GLM four times in a row. Winnable; a real capability gap between cheap models. Its receipts are also where the surface defect showed up live — the flattened-error partial chain above is task 32’s first turn — so the task is both things: winnable under GLM, and the exhibit for what the wire was getting wrong.
• Task 39 is the strange one. The gold requires reservation MSJ4OA cancelled because its record says insurance: yes. But the written policy makes insurance reason-conditional — “full refund … given health or weather reasons” — and the simulated user never supplies a qualifying reason. GLM did the most policy-faithful thing we’ve seen on this task: enumerated all seven reservations, cancelled the two unambiguously eligible, asked the user whether the reason was health or weather, got a non-answer, and refused. Reward: 0.0. Where the eval’s gold encodes a looser reading than the policy document supports, the policy-faithful agent loses — correctly, by its own lights. A 0.0 is not automatically an agent fault; before treating a loss as a surface defect, read the gold against the document.

(The same task under muse produced the opposite failure texture: 100 model calls, 2.25M prompt tokens, ninety-odd refusals-and-reasks before the step cap. GLM did it in five calls and ninety-nine seconds. Neither scored.)

What’s durable

The session’s residue, in order of how much it matters:

1. The interpreter evaluates partial programs — and failure is a branch, not a payload. A reply can run its prefix, bind what it bound, and be refused at the step where the program stops making sense. As of today that refusal has a declared vocabulary too: a failed call fires error on its own branch, and an unarmed error short-circuits with the branch name, the payload, and the held handles on record — the exact report a retry needs. That is a better substrate for the thing kopium is actually for — models defining flows that get replayed and promoted — than a channel where a turn either passes whole or fails whole.
2. Law lives in the system message; action lives in the vocabulary. The domain policy moved from a mid-transcript user message into the byte-stable system prefix beside the grammar. It costs no turn, rides the cache, and — measured honestly — it didn’t flip the losses, because the losses were never placement problems.
3. The ledger is the result, not the leaderboard. Pass/fail alone would have left task 39 an unexplained zero. The receipts — per-step dispatch reports, verdict counters, per-model draws — are what turned it into three different diagnoses.

The open rung is still the recursion: tor name { field: type } plus name = a call or chain declares a flow into the session’s own vocabulary. The promotion gate itself has since been proven end-to-end — headless/probe_promote.k replays a define → stage → baseline → judge → promote → heir arc on canned sessions — but no live model has yet driven that loop itself. That’s the next session’s work, and today bought the instrument to watch it with.

The runtime and harness live at github.com/korulang/kopium; every quote above traces to a receipt under bench/results/runs/compare-glm-*.