The Server Challenge: a Multiplayer Server as Koru's North Star

The SERVER_CHALLENGE is a standing brief: put the sims on the wire — a real socket, an authoritative server, tick-driven — and name what breaks. Not a benchmark suite, not a demo — a forcing function. The package is the instrument; the toolchain is the product. Every wall the consumer hits is a compiler gap to pin, and every gap the compiler can’t cross yet gets named rather than routed around.

Why a multiplayer server

A game server is the least fakeable consumer a language can have. It owns a tick budget it cannot negotiate (16.7ms at 60Hz), transports that lose (UDP drops; the kernel discards what you don’t drain), identity that must stay stable (a player is a pid, not a pointer), and state that must replicate (the world you simulate is not the world anyone sees). Every one of those is a measurable oracle: either the sessions all heard, or they didn’t; either the counter moved, or it’s flat.

And there are others to measure up to — that is half the point. The reference shelf (Quake III, DDNet, Veloren, Cube 2 on ENet, and the rest) is executable — the same bot harness can point at their dedicated servers and ours, and tick cost, bytes-per-snapshot, and players-per-core become byte-exact comparisons instead of vibes. The ambition has a name: WARDOGS-scale — ~100 players, three teams, a control zone, a few hundred store rows. The sim was never the hard part. Replication is.

What the challenge is actually testing

Underneath the netcode there’s a bigger claim this work exists to falsify. Koru isn’t a language with a compiler — it’s a compiler construction kit that happens to have a language-shaped front. A Koru module isn’t code you call; it’s a compile-time machine that reads your program and emits the surface the program needs. std/pump composes two independently blocking systems into one cooperative driver at compile time. std/refine turns declarations into checked constraints. And std/store grew ! first and | none in a day — not a new builtin in a frozen grammar, a new arm vocabulary on a library, because a game server’s input path wanted find-or-join.

That is worth saying carefully, because it’s easy to overclaim: most languages don’t even have a goal that matches this. The nearest neighbor is Racket’s language-oriented programming — a tower of languages for expressing problems — but its goal is DSL construction, not a systems toolchain where every library is a compiler pass and your program is the thing the passes run on. Zig’s comptime is the systems cousin, but comptime computes values; it doesn’t compose machines. Koru’s bet is the second thing — and a multiplayer server is exactly the consumer that will say whether the bet pays: a deadline-paced tick loop, a store that needs an index, a wire format that has to change. Each demand is a question to the toolkit’s end-surface, and each answer lands pinned by an executable oracle or doesn’t land at all.

And the bet already has a measured exhibit, not just a roadmap. std/kernel — declare relationships between data elements, and the transform decides layout and iteration — runs the language-shootout nbody at 1.276s against C’s 1.307, Rust’s 1.389, Zig’s 1.358 (only a hand-tuned scalarized-no-fast-math C reference, 1.227s, stays ahead — named honestly), while the construction-kit cousins sit at SBCL 2.412 and GHC 3.046 in the same table. And the same kernel source, unchanged, lowers to SPIR-V and executes on the GPU of the machine that compiled it — spirv-val gates the build, so an invalid blob is a compile failure, not a shipped one (; the nbody benchmark: ). One declaration, two lowering targets — the “end-surface you need” claim made literal. The named residual: the [gpu] call-site knob is scaffolding; target selection awaits its own selection-language — which is, not coincidentally, the same question the multiplayer declaration asks.

The first replay: asteroids-net

games/asteroids-net is the first replay. Seven programs share one sim and one wire codec:

The wire grew a return channel

input    C→S  "KAI2" | seq u32 | flags u8
snapshot S→C  "KAS4" | tick u32 | count u16 | you u16 | indrop u16
              | count×28B entities | score f32 | wave f32

Two fields in the header are patched per-recipient inside snap.send, which already iterates session rows one datagram at a time:

• you — the join-ack. This datagram’s recipient owns pid you. No handshake datagram: every snapshot is the ack, and a missed one self-heals next tick. The client paints its own ship cyan.
• indrop — how many of this session’s inputs never arrived. seq is the client tick, so a jump in seq counts the gap into drops. Loss and under-send are indistinguishable on UDP — both are “inputs that didn’t arrive” — and the header says so honestly.

One stamp was a trick; two is a channel. “What the server believes about YOUR session” has a return path now, and it cost a patch in the fan-out loop, not a protocol.

The sibling finding: silence is not a client state. A join-then-listen client hears ~90 snaps — the stale sweep reaps it in ~1.5s. Receiving is not participation. The resolution was structural: every input packet runs sim:drive, which touches last — so a periodic zero-flag packet is the keepalive. No PING type exists or needs one.

The probes assert, they don’t observe

$ ./test.sh
server     build... ok
driver     build... ok
watcher    build... ok

watcher: 359 snaps, pid 2 — 1 mine-parked, 1 parked, 1 driving
ISOLATION: PASS
driver: 359 snaps, last count 6, indrop 5
LOSS-ORACLE: PASS

Two oracles, both honest about what they can and can’t see:

• Isolation — the watcher keepalives, learns its pid off snap.you, and tracks ships by owner pid. It asserts its own ship stayed parked while a driven ship moved. A cross-owner input leak fails the first half; dead input fails the second. “Some ship didn’t move” would be a weaker claim — this one is identity-pinned.
• Loss — the driver withholds ticks 140–144 outright. Not a flag, not a simulation — five packets that never left the process. The server counts drops += 5 when seq jumps 139→145 and echoes it back as indrop. Injected vs measured: indrop == 5 or FAIL.

Dup/reorder is the third behavior: seq <= lseq touches liveness but is never re-applied — a stale packet can’t replay old inputs.

The yield: three compiler bugs

This is why the challenge exists. The consumer is the instrument; the toolchain is the product. Each bug below was pinned red first in tests/regression/, then fixed in koru — the game never shapes around a compiler defect:

The third one’s second half is the story worth retelling. The rewrite ran — instrumented, it produced the correct rewritten string — yet the emitted code kept the raw reference. The walk target was a const stack local mutated through @constCast: the write was undefined behavior the compiler was entitled to drop, and it did — the field read back new inside the walk and stale one return later. Heap-aliased argument writes had been landing all along, which is exactly what made const look harmless.

That is a bug class only a real consumer surfaces: nobody writes if(row.field) under a query sweep until a client needs to know which ship is cyan.

Then the probe found the walls

loadbot.k is not a simulation — it is a real client: own process, own ephemeral port, its own session key, sequenced input every tick. load.sh N spawns N of them against one server and asserts every bot heard and the server survived. The escalation ladder, measured:

The N=17 crash was a declared boundary hitting its pinned behavior, not a compiler bug — | full exists and panics precisely so the store’s ceiling is a decision, not a silent loss. The fix is the same arm everywhere: players got capacity 128 and a | full that refuses the join — the refused packet leaves lookup at 0, so its input no-ops and the client simply never hears a pid. Every other insert (fire, rock-split, wave spawn, ship spawn) got the same arm: a full store now drops the entity, never the process.

Wall one: the kernel’s counter

N=100 is the run worth the paragraph. All 100 sessions joined, all heard — and the loss oracle, built to verify injected gaps, caught real loss. Where do the packets die? Not in transit — this is loopback. The kernel counter answers exactly:

netstat -s -p udp, before → after one N=100 run:
    17244 → 35528 dropped due to full socket buffers

+18,284 datagrams discarded by the kernel because the server’s receive buffer overflowed between drains. The drain happens once per tick; the tick body at 100 sessions is a ~120-entity snapshot serialized and sendto‘d 100 times — roughly 340KB of fan-out per tick. When the tick runs long, the burst piles up in the kernel and the counter moves. That is the wall, and it has a name now: ingress drain and full-state fan-out share the same tick budget.

The fix was two compounding causes, and the measurement split them apart. First, packets(max: 64) was a structural deficit: ~100 inputs arrive per tick, at most 64 are drained — the kernel buffer didn’t need to be shallow to overflow, it only needed to be finite. Second, the shallow part: SO_RCVBUF had no surface in the udp lift at all. So the lift grew udp:socket.rcvbuf (set) + udp:socket.rcvsize (read back the granted size — the kernel may clamp, and the log should say what it got, not what was asked), the server asks 4MB and gets 4MB, and the drain cap moved to 512 — above the 128-session ceiling.

The rerun is the whole payoff, one line of netstat: 35,528 → 35,528. The counter did not move. indrop 0 across all 100 sessions, 44,024 packets in (+53% ingress), every bot still heard. The wall wasn’t “UDP can’t do 100 players” — it was a 64-per-tick drain cap and a socket option nobody had needed yet.

The ladder, asserted not sampled

Then the parting theory — fan-out starves the drain — got measured instead of assumed, and the probe went 8 → 1,024 sessions at 59.9Hz with zero measured loss. Every wall it hit was a different kind than the one before.

Wall two, disproven: the fan-out wasn’t the drain

Timers at four points inside handle (entry, post-accessors, post-decode, post-sim:drive) plus a whole-packets-step total decompose the tick at N=128:

drain 496.6ms/beat = hdr 3.96 + dec 8.87 + drive 334.1 + disp 149.6
sim ~0.1ms/tick, bcast ~0.9ms/tick, drain ~0.83ms/tick
per packet: drive 6.6us + dispatch 2.9us + decode 0.17us + accessors 0.08us

Per packet the cost is ~10µs — sim:drive (two store scans over players and bodies) plus arm dispatch. Decode is noise; the fixed-layout codec earns its keep. Fan-out is real (~0.9ms/tick) but not the wall — at 59.9Hz the tick body totals ~1.9ms against a 16.67ms budget.

The feature that shipped anyway: ! first and | none

The per-packet scans were the suspect, and the fix deserved to exist regardless: std/store:query gained ! first <row> when <cond> — the early-exit sibling of ! query — and | none, the absent case, which is what find-or-join actually wants:

// sim/index.k — drive is the whole input path: find-or-join in one arm pair
drive = std/store:query(players)
    ! first p when p.key == key
        |> if (seq > p.lseq)
            | then |> std/store:stored { p.last: tick, p.drops: p.drops + seq - p.lseq - 1.0, p.lseq: seq }
                |> control(p.pid, left, right, thrusting)
                |> fire(p.pid, pressed: firing)
            | else |> std/store:stored { p.last: tick }
    | none |> std/grid:stored { lookup[0].pid: 0.0 }
        |> spawn-player(key, tick, seq)
        |> if (lookup[0].pid != 0.0)
            | then |> control(lookup[0].pid, left, right, thrusting)
                |> fire(lookup[0].pid, pressed: firing)
            | else |> _

The arm pair deleted an entire protocol: find-touch, fresh-pid, and a lookup grid column whose only job was ferrying “did we just join?” between two store scans. The pins: (early exit, absent arm) and (stored writes and captures in both arm shapes).

Then the honest part: the A/B falsified the scan-cost hypothesis. Under matched load, first/none measured 41.6/40.9Hz against the old code’s 40.8Hz — neutral. The scans were never the cost; the feature ships for expressiveness, which is the better reason.

Wall three: the clock, not the work

With everything inside the tick accounted for (~1.9ms of work) and the server still sagging to ~48Hz, the whole-tick-body timer found what the phase timers couldn’t: std/time:sleep-ms(16) slept ~18.7ms. Nanosleep quantization plus wake overshoot — invisible to every timer that only measures work, because it happens between work.

The fix already existed, landed the same morning by a parallel session: std/time:pace(period, last) — deadline pacing, which sleeps until last + period on the schedule rather than sleeping a duration after work finishes. One-line swap; sleep accounting dropped to ~14ms/tick and the loop went 48.6 → 59.9Hz.

Wall four: a u8 on the wire

N=256 didn’t degrade — it crashed, mid-broadcast:

integer part of floating point value out of bounds
  at @intFromFloat, snap.entity → sendto

you was a u8 on the wire, and pid 256 is the first that doesn’t fit a byte. Every run at N≤255 was green; the bug was invisible until the exact boundary. Entity owner had the same latent defect.

KAS4 spends the pad bytes v0 reserved — you u8→u16 in the header, owner u8→u16 in the entity tail — packet sizes unchanged, magic bumped, readers’ offsets updated in one file. The snap buffer went 8192→65536, which is now literally the UDP payload ceiling.

The ladder’s top rung

~70K inputs/sec and ~87K datagrams/sec through one thread, at the declared rate, with measured work at ~4× headroom under the tick budget.

Wall five: arithmetic

The next wall can’t be bought off. A UDP datagram maxes at 65,507 bytes; a KAS4 entity is 28 bytes → ~2,340 entities fit one datagram. At ~N=1,500+ (world density depending), the world stops being expressible in a single snapshot — full-state fan-out doesn’t get slow, it gets impossible. That’s where delta snapshots and interest management stop being optimizations and become the only option.

The wall is now instrumented, not silent: snap.entity/snap.trailer used to early-return when the buffer filled — the world would have shrunk with no signal. They count into snap.truncated, and the beat prints SNAP TRUNCATED the moment an entity doesn’t fit. And deltas need what the store design had been circling: stable entity identity (swap-remove churns positions; a per-recipient baseline can’t diff by slot) — which is what made declared indexed columns the load-bearing ruling. That ruling has now landed — the index organ is its own post, and its first consumer is this server’s session store.

What the challenge pushed

The consumer is the instrument; the toolchain is the product. The count so far:

The honest detail worth keeping: ! first measured performance-neutral — it shipped for expressiveness, which is the better reason, and the A/B that proved neutrality is the kind of oracle the challenge exists to supply.

The road, named not promised

The standing brief’s gap list is the north star — each item a known absence, in roughly the order the walls will be met:

• Conjunct routing — the flat index measurement’s named next rung: split e.owner == pid out of compound guards, route it to a bodies.owner index, evaluate the residual on the resolved row. Where the drive number actually lives.
• Delta snapshots + interest management — forced, not optional: the datagram can’t hold the world past ~N=1,500.
• Loss repair — indrop measures; acks/resends and client response are unbuilt.
• Session lifecycle, prediction, lag-compensated rewind — phantom machinery in its intended habitat, named in the brief, not yet pressed.
• games/zonecontrol — the boss target: three-team KOTH at WARDOGS shape, where the rules are a weekend of store queries and the feel layer stays honestly out of reach.
• Signal/telemetry — the same tick loop feeding koru/signal detectors: tick_ms over budget is the first model; per-player physics-consistency detectors later. The anti-cheat half of the ambition.

What the surface looks like

The two-tier shape all of this is aimed at: an indie developer writes ~400 lines declaring what their game is — transport, wire, sim, and the invariants their game asserts — and the composition emits the backend this post just measured. Not a framework you call into; a pipeline the compiler composes. A shape sketch, deliberately not a spelling (spellings get ruled, not sketched):

server = a multiplayer declaration naming
    transport (udp) · wire format (KAS4) · the sim
    · integrity invariants (input-rate, physics-bound, …)

client = the same game's declaration, other end:
    predict (your own ship) · interpolate (remote entities)
    · paint (raylib, or later: canvas) · transport

The integrity line deserves its own paragraph, because it’s the clearest case of the thesis: anti-cheat isn’t a thing you ship — it’s something a game expresses. Chess is cheated by statistical violation (move-quality distributions that don’t match a human); poker by information violation (collusion — knowledge crossing accounts); a shooter by physics violation (state transitions outside the human envelope). Three different things “integrity” means — so the honest surface is a vocabulary for declaring what cheating means in your game, and koru/signal is that vocabulary, not a product. For asteroids-net the boundary already deleted the worst cheat class: clients send intent, never state, so “my position” isn’t a field that exists on the wire. What remains — input abuse, information advantage, physics violations — maps one-to-one onto organs the roadmap already names, and the physics oracle is the sim itself: the server can check any claimed effect against rules it already runs.

The frontend is the symmetric half, and it’s further along than it looks: one sim already runs under five shells — the server, the raylib client, the watcher, the driver, the loadbot. The client pipeline mirrors the server’s — input → predict → interpolate → paint — and prediction policy is game-shaped the same way integrity is (chess predicts nothing; a shooter predicts nearly everything). The one honest hard problem: the server owns a single timeline while the client owns three — predicted state (now), last-authoritative state (then), interpolated render positions (in between) — so a client declaration is temporal policy more than stage order, and that’s the part that has to be sketched against client.k’s real shape before it can be trusted. The ending worth wanting: the sim’s JS emit road is real, so the same declaration is one transport-and-paint lift away from a browser client — a page on this site that is a live client of the running server.

What this is not

Cautious, deliberately: this is a forcing function with good numbers, not a netcode product. There is no loss repair yet, no client prediction, no auth — session identity is addr<<16|port, fine for loopback, not for the internet. The 59.9Hz figure is one thread on one machine under a synthetic bot load; the executable-oracle comparisons against real servers haven’t run yet. And the moment a wall turns out to be a compiler defect, consumer work stops and the fix lands in koru first — that discipline is the point, and it has already held more than once.

The brief and its done-gates live at SERVER_CHALLENGE.md; the replay log at challenges/server/CATALOG.md. The slice runs: cd games/asteroids-net && ./test.sh — and ./load.sh N when you want to watch a wall get found. The index organ that grew out of it: std/indexes.