A 3-bit Qwen3.8-Flash-Next on a single local GPU, running in the open-source Rameness harness, built a browser Minecraft clone that passes all 14 behaviour checks, the same as Claude Code with Claude Opus 5.5. Two weeks earlier, Rameness scored 0% on the same task; this is what changed.
What Rameness is
Rameness is an agent harness: the control system around an LLM that gives it work, chooses what should happen next, runs tools, checks real results and recovers when something fails. It separates small, typed control decisions from the open-ended work of reasoning and writing code, then feeds what actually happened back into the next step.
The left side is a deliberately simplified agent loop. The right side shows the extra control and feedback paths this post means when it says “Rameness.”
The benchmark
One prompt, an empty folder, 150 minutes: “Build a Minecraft clone that runs in the browser.” The result must work when index.html is served as static files, with no build step.
- Behaviour: 14 scripted checks in a real browser. Does it load and render, can you move, look around with the mouse, jump, break and place blocks? The score is the percentage passed.
- Visual: a Claude judge grades one final screenshot from 0 to 10.
- No web search: the Rameness runs had no working web search, so the agent worked from what the model knows about Minecraft, 3D graphics and browser APIs, plus what it could run and read locally.
It’s a deliberately long, open-ended task. The agent has to plan, write thousands of lines across many files, test its own work in a browser, and recover from its own bugs, all without a person in the loop.
Results
Two local Qwen models, in several builds, passed all 14 checks in Rameness, the same as Claude Code with Claude Opus 5.5. Opus did it in under 14 minutes; the local runs used most of their 150.
| Harness + model | Behaviour (14 checks) | Visual (out of 10) |
|---|---|---|
| Claude Code + Claude Opus 5.5 (reference, 13.7 min) | 100% | 7.25 |
| Rameness + Qwen3.8-Flash-Next, 3-bit AutoRound, Clef deciding | 100% | 8.0 human review (6.5 judge) |
| Rameness + Qwen3.8-Flash-Next, 3-bit AutoRound, Kev deciding | 100% | 7.5 |
| Rameness + Qwen3.8-27B, 4-bit (llama.cpp) | 100% | 7.25 |
| Rameness + Qwen3.8-27B, GPTQ (vLLM), 2 runs | 79–100% | 6.75 |
| Claude Code + Qwen3.8-27B, 2 runs | 93–100% | 3.75–5.5 |
| Rameness + occamy-1.0 (35B-A3B, 3.4-bit) | 14–86% | 0–4.5 |
| Claude Code + occamy-1.0, 1 run | 64% | 3.25 |
The same model did better in Rameness than in Claude Code: Qwen3.8-27B scored the same on behaviour in both, but Rameness’s builds looked better, and occamy did better in Rameness’s best runs. The Clef build’s 8.0 is a human grade: the automated judge saw one screenshot of a savanna coast, where that build’s world rarely places trees, and gave 6.5. Every other visual grade is the judge’s.
From 0% to 100% in two weeks
On 2026-09-25, Rameness scored 0% on this task. By 2026-10-07 it passed every check. Most of the gain came from three changes to how the agent works, not from bigger models.
| Date | Change | Effect |
|---|---|---|
| 2026-09-25 | Baseline | Rameness 0% with Qwen3.8-27B; Claude Code with the same model 92.9% |
| 2026-09-25 | The agent writes its own test plan and keeps a todo list | Qwen3.8-27B to 85.7% |
| 2026-09-26 | Syntax diagnostics after every file write | occamy to 85.7% |
| 2026-09-26 | The stand-in rule: a check that replaces a part with a stub or mock proves nothing about that part, and the self-review asks the same | First 100% (Qwen3.8-27B, visual 6.0) |
| 2026-10-07 | Draft first, then revise: a complete, running draft of the whole game before perfecting any part, then rounds of review-all, fix-all, run-all | Flash-Next from 64.3% (no runnable game until minute 121) to 92.9% (a page from minute 1) |
| 2026-10-09 | The scorer fixed: it now holds the mouse to mine, as Minecraft does, instead of clicking once | The Flash-Next build rescored from 92.9% to 100% |
The stand-in rule was the turning point. A run before it had built a test rig with WebGL stubbed out, saw its tests pass, and shipped an invisible world. With the rule, the model went looking for a real browser on its own at minute 16 and checked real rendered pixels from then on; the prompt never mentions browsers. The draft-first rule came from watching how Opus built the game in Claude Code: 26 turns that got a whole, running game early, against Rameness’s 263 turns of polishing parts.
The last row is a reminder that benchmarks have bugs too. A faithful Minecraft clone mines by holding the button, and the old check failed it.
Running it locally
The best local runs used Qwen3.8-Flash-Next, quantized to 3 bits with AutoRound, on a single local GPU. It ran on a patched vLLM with multi-token prediction and a 262k-token context window. The decision model ran separately: Kev on the CPU, later Clef on an RTX 5070 Ti.
Rameness benchmark data from docs/benchmarks.md. The 27B line shows median tokens per second in each 25k context band of one run; the Flash-Next annotation summarizes its sparse-attention speed tests.
Speed over a long context is what made Flash-Next the better fit. A Minecraft build fills the context with code and test output, and a dense model reads its whole cache at every step, so it slows as the run goes on. Flash-Next’s sparse attention reads a fixed 2,048 tokens, so its speed held.
Swapping the decision model: Kev vs Clef on the same build
Rameness’s small decision model (JEV) chooses, among other things, how hard the LLM thinks on each turn. We ran the same model on the same build with Kev-4B deciding, then with Clef-Flash. Both passed all 14 checks.
| Kev-4B (CPU) | Clef-Flash (GPU) | |
|---|---|---|
| Time per decision (median) | 1.50 s | 0.18 s |
| Thinking level chosen most | minimal, on 50% of turns | maximum, on 34% of turns |
| Model calls / tokens written / time generating | 489 / 664k / 103 min | 628 / 577k / 92 min |
| Game code / its own tests | 4,166 / 2,314 lines | 4,485 / 2,677 lines |
| Minecraft features (of a 56-item checklist) | 47 | 48 (53 with a round finished too late to merge) |
| Biomes | 5 | 9, chosen by temperature and humidity like the real game |
| Where the time went | about 45 min on one rendering bug | core done by minute 69, then a feature round |
With Clef, the model thought hard more often yet generated 13% fewer tokens, and wrote 8% more game code and 16% more tests (25% and 61% counting the round that missed the deadline). These are single runs two days apart with harness changes between them; the decision-model comparison is the controlled test.
Caveats
- One run is a sample, not a verdict. Identical code on the same model has scored 35.7% and 78.6%. The best results here are single runs.
- One screenshot sets the visual grade. Each game generates a new world on load, so the judge can land on a bare coast or a forest.
- Older runs weren’t rescored. 63 archived runs failed the old mining check; only the Flash-Next build was rescored, so other scores may be one check low.
- Time isn’t comparable. Opus finished in 13.7 minutes; the local runs had a 150-minute budget and used most of it.
Every run’s score, judge notes, event log and model-request log are in the Rameness repository under bench/minecraft/runs, with the version-by-version history in docs/iterations.md.