related_post_gen: Koru Beats Every Rust Entry

All three of them — standard, highly-optimized, and the parallel one. On a laptop, not the leaderboard.

October 3, 2026 · 9 min read

TL;DR

related_post_gen is a public benchmark: read posts.json (up to 100k posts, up to 100 tags each), compute the five most-related posts for every post by shared-tag count, write the result as JSON. The repo ships a correctness oracle (correct_related.json + verify.py) and times the whole process — parse, compute, serialize, write — with GNU time. We wrote a Koru entry and raced it against all three Rust implementations in the repo, interleaved, minimum-of-N, on this machine (Apple M2 Pro, single-threaded throughout):

	5,000 posts	20,000 posts	60,000 posts
Koru (this entry)	10.9 ms	37.4 ms	114.6 ms
Rust HO — powerset intersections	13.9 ms	43.9 ms	133.0 ms
Rust con — rayon, parallel	12.3 ms	49.8 ms	244.3 ms
Rust — standard entry	20.3 ms	170.9 ms	1226.2 ms

Whole-process wall clock, minimum of interleaved runs, one Apple M2 Pro. Three panels, three scales — the 5k and 60k columns differ by two orders of magnitude and would lie on one axis.

vs. the fastest Rust entry — rust_ho, single-threaded like ours — that is 0.78x at 5k, 0.85x at 20k, 0.86x at 60k. We also beat rust_con, which distributes the scoring loop across all cores with rayon and still loses to a single Koru thread at every size.

Three honest scopings before the story. One: these are laptop numbers on this machine, not the repo’s leaderboard column — their numbers come from a dedicated-CPU AWS box, and only a submission would land there. Two: the benchmark’s rules ban FFI, and our JSON parser is koru/yyjson, a packaged binding to the C library — the same role serde_json plays for Rust, but a rules-read we haven’t argued and won’t here; measured parse times came out about equal anyway. Three: the win is not “our scoring loop is faster” — rust_ho’s core still beats ours on the same-instrument number. Where the margin actually lives is below.

The program

The entry is a Koru program (main.kz, 566 lines) — flow code orchestrating the whole computation, |zig leaf tors only where the surface ends: byte buffers, flat arrays, packed keys, serialization. The algorithm is the powerset-intersection scheme the repo’s own rust_ho entry uses, written in Koru’s flow rather than pasted as a Zig blob:

Intern every tag through std/string-map into an integer id, building a flat CSR of post → tag-ids while stashing each post’s yyjson *Value pointer so nothing ever re-walks the document.
For cardinality k from k_max down to 1 — the loop is the program, in flow:

|> ieach(lo: 1, hi: kmax1)
! each i |> isub(a: kmax1, b: i): kk
    |> ia.get(a: ucell, i: 0): u
    |> if(u > 0)
        | then |> ia.set(a: ncell, i: 0, v: 0)
            |> ieach(lo: 0, hi: np)
            ! each p |> pset.post(tag_ids, tag_off, keys, cands, ncell, p, k: kk)
            | done |> ia.get(a: ncell, i: 0): npair
                |> pairs.sort(keys, cands, n: npair)
                |> dissem.run(keys, cands, n: npair, ptot, topns): unres
                |> ia.set(a: ucell, i: 0, v: unres)

Each iteration enumerates every k-subset of every post’s tag list packed into a u64 key (pset.post), sorts the pairs (pairs.sort), and for every group of ≥2 posts sharing a subset gives each member the others as candidates (dissem.run), stopping when every post has its five — the if(u > 0) gate is the early exit, in the language, not in a leaf.

Emit is one serializer leaf writing borrowed strings into a preallocated buffer, then one write().

Two declared limits: a post with more than 8 tags, or more than 256 distinct tags overall, aborts loudly — packed keys and the tag-id width can’t express those inputs, and the standard-algorithm fallback rust_ho carries for that case isn’t wired into ours yet. Loud failure, not a silent wrong answer; the benchmark’s own data (4–5 tags, 28 distinct) never approaches either bound.

Where the win actually lives

Not the algorithmic core. The binaries print the same instrument — Processing time (w/o IO), spanning post-parse indexing + scoring, before serialization — and at 5k the mins are rust_ho 3.47 ms, Koru 5.66 ms. Their core is ~1.6x faster than ours, honestly.

The whole-process margin comes from the ends of the program:

Parse is a wash. A read+parse-only probe of each side: Koru 3.8 ms whole-process, Rust 4.1 ms — process startup noise dominates both; serde_json and yyjson are within noise of each other at this size.
Serialize+write is the whole ballgame at small n. A probe running the standard entry’s exact serde_json::to_writer path measured 5.24 ms at 5k — over a third of rust_ho’s whole-process time, before output. Our equivalent is one fused leaf per post copying borrowed string bytes into a buffer, plus a single write(): under a millisecond end to end, and it scales linearly while the oracle stays green.

So the honest sentence is: Koru’s emit path is fast enough that a slower core still wins the process. At 60k the gap widens — 114.6 vs 133.0 — because serde’s serializer pays per-field machinery on 26.3 MB of output while ours copies bytes.

What it took to get here — including the wrong turn

The first version of the program used the standard algorithm — the one in the rust/ entry: scatter shared-tag counts into a dense u8 array per post, scan for the top five, re-zero. That version beat rust/ on whole process (0.78x) after two real fixes: discovering yyjson_arr_get is O(index) on non-flat arrays — every post contains a nested tag array, so repeated array.get(root, i) was quadratic, 12.5M element-skips per pass, fixed by stashing post pointers once — and replacing a scalar 64-byte chunk check in the top-5 scan with @reduce(.Max, @Vector(64, u8)).

A touched-index list — only rescan slots that were bumped — was the one idea that lost: with 28 distinct tags nearly every post shares a tag with nearly every other, so the first-touch branch mispredicted constantly and the “sparse” list was the whole array. 45 ms versus 20 ms. Reverted.

Then we raced the rest of the repo — which we should have done first, and didn’t — and rust_ho beat the scatter version outright (13.6 ms vs 15.7 ms) with a different algorithm entirely. So we ported that: per-post k-subset enumeration into packed u64 keys, pair sort, disseminate, early exit — and that’s the program in the table. The powerset path also dropped peak RSS from 8.4 MB to 1.5 MB against rust_ho’s 5.7 MB (/usr/bin/time -l peak resident, bytes→MB), and the binary is 89 KB versus their 402 KB — size wins we didn’t engineer for, they fell out of not carrying a serde derive pipeline.

The one thing we will not claim: source size. 566 lines to rust_ho’s 369 — ~190 of ours are the |zig leaf plumbing for flat arrays and byte buffers, the cost of doing this in a language whose std surface is still being built.

Where this leaves us

A Koru program on a real, externally-oracled benchmark beats every Rust entry in the repo — including the parallel one — by enough that laptop noise isn’t the story (0.78–0.86x across three scales, verified output each run). The remaining honest gaps: a submission needs the k_max > 8 fallback wired (the spec allows 100 tags per post; our data has 5), a real leaderboard number needs their dedicated-hardware run, and the FFI reading of koru/yyjson would need a maintainer’s ruling or a native Koru JSON parser — which our std surface census already wanted anyway.

The program, the generated datasets, and the race harness are in the benchmark checkout; nothing here is submitted to the benchmark repo.