Packing Arena
A reproducible one-dimensional bin-packing tournament where three independent AI coding agents (Scarlet, Cobalt, and Jade) implement deterministic solvers, perform circular code review with concrete counterexamples, revise once, and compete on a pre-committed 108-case synthetic benchmark.
What I built
A zero-dependency Python benchmarking harness and interactive static results explorer designed to test multi-agent algorithm design, adversarial code critique, and strict verification gates on 1D bin packing (capacity 100, 0–180 items per case):
- Sealed Pre-Committed Dataset (
cases.json): 108 synthetic instances across 8 families (edge,uniform,small,large,near_third,complements,mixed, and 24exact_smallcases with proven optima), locked by SHA-256 (ffa9821b...) before any peer review or revision. - Three Deterministic Solvers & Circular Peer Review: Each agent combined Best-Fit Decreasing (BFD), bitset subset-sum completion, L2/cardinality lower bounds, and operation-budgeted exact search. In the review round, Scarlet constructed a 60-triple counterexample where Cobalt used 61 bins; Cobalt identified an early
depth >= 600cutoff in Jade (57 vs. 55 bins); and Jade exposed missing cardinality lower bounds in Scarlet. - Deterministic Judge & Independent Oracle (
judge.py,verify.py): Evaluates 1,944 total calls (108 cases × 3 agents × 2 rounds × 3 rotated repetitions), checking index completeness, bin capacity, non-mutation of input arrays, output determinism across repetitions, and agreement with an exhaustive branch-and-bound oracle cross-checked on 60 random cases. - Interactive Results Dashboard (
index.html): Dependency-free browser explorer comparing pre-review (`round1/`) and post-review packings, family-level bin totals against the 3,333-bin BFD baseline, per-bin visual item bars, full peer-review transcripts, and frozen source code.
What was checked
Release audit: 26 October 2026 (commit e417e3f1, release v0.1.0). Both Jade (0.135 ms median) and Scarlet (0.251 ms median) reduced total bins from the 3,333 BFD baseline to 3,321 with zero failures and 29/29 exact-small optimum hits. Cobalt achieved the same 3,321 output bins but exceeded the 1-second limit on case C017 (1,632.29 ms) after stacking pattern search ahead of unbucketed list operations. Verified by 4 unittest suites and 648 saved-packing checks on Linux CI (Python 3.10 & 3.14) and Windows.
What this demonstrates
Reproducible benchmarking, pre-committed dataset hashing, independent oracle verification, and transparent reporting of both improvements and regressions (such as Cobalt's post-revision timeout). All test cases are synthetic, and the three contestants are sub-agents of the same parent model rather than separate model families. Wall-clock tie-breaker measurements reflect the recorded Windows run; CI verifies algorithmic correctness and frozen evidence hashes.