Build Your Own Faulty Compiler

Part 2 of 2. Companion to The Faulty Compiler.

A fun way to understand the mind behind the curtain of this new technology is to build your own harness.

If you’re a developer, one of the best things you can do with modest compute is write an agentic coding harness: the loop that hands a model a task, gives it tools, and feeds the results back in. Use your favorite frontier model and coding tool to write it, in whatever language you like. Then point it at a small local model and give that model real work. Here are a few decisions to make:

• Local LLM framework: Ollama is a decent choice.
• Models: Gemma 4 and Qwen3-Coder both support tool calling. gemma4:12b is about 8 GB and qwen3-coder:30b is about 19 GB. Use whatever is current when you read this.
• Harness language: Python, Rust, Go.
• Test project: You need something for the model to do, after all.

A small note on the test project: make sure it has tests. It makes the adventure more fun.

This experiment does not reproduce frontier capability, and a small local model is not a stand-in for one. It exposes the engineering categories hidden by capability claims: retries, guards, verification, scaffolding, and resource consumption. Those are the things to measure.

It should take you down a pretty decent rabbit hole. While you’re falling, Alice, I want you to catalogue your observations about building your new toy, what they portend about our actual course, and what they say about the people steering this revolution.

1. Your coding agent likely won’t one-shot a universal harness. By universal I mean one that works across whatever models you point it at. Given a good prompt and requirements, it will build a functional harness that calls and interoperates with the model. Very impressive: that used to take a junior engineer a few weeks. But really, we’re talking about an API or two and some shims for various tools. Not amazingly hard. So if AI is ready to take over systems at will, why can’t it do this out of the box with ZERO issues? I’m not guessing. I’ve built two harnesses with Claude, and neither worked without a ton of my input and a lot of iterations. The build notes at the end list what had to be added, and nearly all of it was learned by watching runs fail, not written up front.
   
   Don’t expect the leaderboards to settle that for you, in either direction. A year ago the best model scored 23% on SWE-Bench Pro. Trackers now show the leaders above 80%, and in between OpenAI found roughly 30% of the benchmark’s tasks broken and withdrew its recommendation to use it.¹ The famous finding that AI tools made experienced developers 19% slower? The lab that ran the study now calls it outdated, and says its newer data is unreliable because developers increasingly refuse to work without AI.² The scoreboard is a faulty compiler too. Build the thing and measure it yourself.
2. Sandbox it and back up before the first run. Your new toy is very faulty, and the harness you just built has to give it some level of CRUD (create, read, update, delete) over your files. A container with no network, a non-root user, and only the project directory mounted is enough. Your first runs will be an experiment in frustration and fascination. You’ll see real wisps of comprehension and of the ability to follow instructions. That amusement is soon dashed against the rocks of specificity, lost context, and a fairly interesting loss of interest in the stated objective. So sandbox no matter what, whether through containers or libraries, and keep a backup you can restore from. You never know which tools the model will reach for, or why.
3. Watch for the nuclear option. Hopefully you’ve steered your agent toward the best local models for your available compute. I hope your compute is limited, so you get to sample the range: tool use, reasoning, and, most importantly, willingness to solve a problem with the nuclear option. That means deleting failing tests, rewriting assertions until they pass, or a git reset --hard. This is why your test project needs tests: tampering with them is the classic tell. Part 1 has the frontier-scale versions of the same behavior.
4. Watch the clock, then run it again. You’ll be thrilled once you’ve tuned your tools and prompts and the harness starts making real progress on your objectives. Depending on how many objectives you have and how complex they are, the wall clock may surprise you. You may be further disappointed when the same model, rerun, doesn’t succeed on the next round of fun. Don’t worry: you’ll get to explore temperature, maybe the seed (check that it’s actually being applied; mine wasn’t), and every one of those experiments will remind you that these less sophisticated models are shadows of the epic, unbridled power and expertise of their big-brother frontier models. Then you’ll recall that your harness and a few simple logical validations are doing more to get you to the finish line than the model is.

Is that a false equivalence? The frontier models are better, after all: refined, perfected. Refined, yes. Perfected, no. But that’s not what the benchmarks say! Then look closely at how the same model does inside and outside its custom harness. The results vary, and far more than they would if the model were perfected and carrying all the water. Researchers now argue that among comparable frontier models, the harness is often a stronger determinant of performance than the model it wraps.³ Keep that in mind when you reread the cyber-attack claims in Part 1. Those results came from a model, a scaffold, and a thousand runs.⁴

When you climb back out of the rabbit hole, you’ll have something the headlines don’t: your own numbers. How many runs it took. How long the clock ran. How much of the result came from the model, and how much from the harness and the checks you wrapped around it. Those are the figures I asked the labs for in Part 1. If you can log them on a laptop, a frontier lab can publish them.

Bringing it home

Put the two parts together. What gets sold as a model’s capability is the capability of a system: the model, the harness, the checks, the reruns, and the compute to pay for all of it. The results get published. The spec sheet doesn’t.

Man, that lossy interpreter has come a long way, all the way to a functional autonomous coding agent. You only had to harness it, give it tools that also guard it from itself, sprinkle in unit tests to keep it focused, and maybe add a linter to stop it drifting. The result is a tool that could never have existed five years ago. Yet it’s still lumpy and jagged. The model alone isn’t coming for you. Institutions using it to cut labor costs already are, and the hand-waving makes those choices easier to defend.

The missing spec sheet matters most for the threat everyone leads with. Cyber is the most obvious risk, and I don’t dismiss it. It is also the one risk on my list with a standardized, repeatable repair loop. The tools that find the holes find them for the defenders too, and the companies and maintainers who ship the code have every reason to close them. The Linux kernel doesn’t assign a CVE until the fix is in a stable release, and it is now approaching 2,000 of them per release.⁵ Google’s CodeMender upstreamed 72 security fixes in its first six months, and DARPA’s finalists were finding and patching real bugs for about $152 a task.⁶ The loop is strained. Of roughly 530 serious bugs Anthropic’s Glasswing project reported to open-source maintainers, 75 had been patched six weeks in, and some maintainers asked it to slow down.⁷ But the loop exists, and money and attention are pouring into it. A patch does not undo stolen data, physical disruption, or intelligence loss, but it can close the hole against the next attack.

Nothing else on that list has a comparably fast, standardized remediation loop. There is no equivalent of coordinated vulnerability disclosure for a released pathogen, a completed targeting decision, a successful influence campaign, or a lost early-career cohort. Countermeasures exist, but the safeguards usually act before or during the harm; they do not restore the world afterward. Those harms are documented today and they don’t need AGI. They are also going largely unaddressed. Cyber may be the most obvious risk, but these are the ones whose damage is hardest to reverse. That is why I think they deserve more of the worry than they get.

Here’s the part the layoffs miss. Every fix in the build notes below came from an engineer watching a model fail and building the guard that caught it. The faulty compiler works because someone verifies it. We are marking down the very expertise the technology depends on. Sure, I can loop this toy, or a frontier model, against tests and verified answers until it generates something that may work. But if we build the next level of everything with these faulty compilers, with little or no supervision and no comprehension, we will have agreed that we do not care about the future.

So I’ll ask again for what I’d ask of any vendor: the specs, the test conditions, and the failure rates. Then spend the worry where there is no patch. None of that is exotic. It is honesty, transparency, and accountability.

Build notes, for those who want to cheat

Here are representative fixes my own harness needed before small local models could get through real work. They come from a stretch of twenty-one pull requests. Every item below is a fix for something a model, or the plumbing around it, actually did.

Tool calls and provider correctness

• <function=name> syntax fallback (PR #46). qwen3-coder emitted this non-JSON tool-call format and the harness extracted nothing, looping 100+ turns with zero tool calls.
• frequency_penalty and seed moved to top-level request fields (PR #48). A model fell into verbatim repetition loops. The setting meant to curb it went through Ollama’s options object, which Ollama’s /v1 endpoint silently ignores. seed had the same bug, so determinism had never actually worked.

Stall and hang detection

• Escalating no-tool-call nudge (PR #51). ministral-3:8b got stuck for 7 turns re-describing an edit plan in prose with zero tool calls, receiving the same static reminder each time.
• Per-chunk LLM-stream watchdog (PR #56). glm-4.7-flash hung for 30+ minutes on a turn that produced zero stream chunks. The existing retry logic only fires on an actual error, so it couldn’t see this failure at all.

The objectives system and its failure modes

• Reject completions whose summary doesn’t match the current objective (PR #54). glm-4.7-flash called complete_objective four times on one real piece of work, silently marking three unrelated objectives “done.”
• Gate complete_objective on a fresh, whole-project reverification (PR #55). glm-4.7-flash narrowed its own verification (a passing scoped ruff check instead of the full one) before claiming completion.
• Harness-native lint tool that hard-blocks both complete_objective and the FINISH-time check (PR #64, reordered in #65). gemma4:e4b left three near-identical dead return statements stacked in one function. The tests kept passing throughout, because the first return always fires, so the existing test gate had nothing to object to. The fix adds an optional lint convention that mirrors test exactly and runs before it, since a static check is cheap and a full test run can legitimately take the whole timeout or hang outright. It has already caught a real bug: in round 4 the lint gate blocked a completion on a genuine F821 undefined-name error, and the model understood the message and started fixing it instead of slipping a broken completion through.

Repeated-failure escalation (the throughline of the whole build)

• Test-timeout infinite-loop nudge (PR #60). gemma4:e4b hit the same 200s test timeout repeatedly, and the harness treated it like an ordinary failure. The root cause was a genuine infinite loop the model had introduced.
• Configurable timeout ceiling (PR #61). gemma4:e4b, stuck on that same hang, started raising timeout_seconds on its own toward a 6-hour wait instead of fixing the bug.
• Same-test-failure streak nudge (PR #63). gemma4:e4b failed one assertion 26 times in a row, with a successful but ineffective edit between every attempt. Each edit reset every existing counter before it could escalate.

The omitted fixes are less dramatic variations on the same themes: malformed tool envelopes, restore semantics, timeout backstops, failure counters, and model-capability discovery. The point is not that every agent will reproduce this exact list. It is that the apparently autonomous result depended on an engineer recognizing each failure and turning it into a guardrail.

---

Sources and notes

1. “SWE-Bench Pro” (arXiv, Sep 2025): “Performance on SWE-Bench PRO remains below 25% (Pass@1), with GPT-5 achieving the highest score to date at 23.3%.” arxiv.org — BenchLM leaderboard tracker (Oct 1, 2026; a secondary aggregator) lists a top score of 89.9% and four more above 80%: benchlm.ai — OpenAI, “Separating signal from noise in coding evaluations” (Jul 8, 2026): “~30% of SWE-bench Pro tasks are broken”; “we retract our earlier recommendation to adopt SWE-Bench Pro.” openai.com ↩
2. METR, Becker et al., “Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity” (Jul 2025): 16 developers, 246 tasks; AI increased completion time by 19%. metr.org — METR, “We are Changing our Developer Productivity Experiment Design” (Feb 24, 2026): newer estimates point toward a speedup, with confidence intervals that still include zero, and “30% to 50% of developers told us that they were choosing not to submit some tasks because they did not want to do them without AI.” metr.org ↩
3. Zhang, Wang, Ge, Xu, Hamm & Reddy, “Stop Comparing LLM Agents Without Disclosing the Harness” (arXiv, May 7, 2026), a position paper: “for long-horizon tasks evaluated across models with comparable frontier capability, the agent execution harness… is often a stronger determinant of agent performance than the model it wraps.” arxiv.org ↩
4. Anthropic, “Assessing Claude Mythos Preview’s cybersecurity capabilities” (Apr 7, 2026): “across a thousand runs through our scaffold, the total cost was under $20,000.” anthropic.com ↩
5. Linux kernel documentation, “CVEs”: “No CVEs will be automatically assigned for unfixed security issues in the Linux kernel”; assignment “will only automatically happen after a fix is available and applied to a stable kernel tree.” docs.kernel.org — Security Boulevard (Sep 2026, a secondary report): about 500 CVEs per release in the 6.x era, approaching 2,000 by Linux 7.3, attributed largely to AI-assisted scanning. securityboulevard.com ↩
6. Google DeepMind, “Introducing CodeMender” (Oct 6, 2025): “we have already upstreamed 72 security fixes to open source projects, including some as large as 4.5 million lines of code.” deepmind.google — DARPA, AIxCC results (Aug 8, 2025): “an average cost per competition task of about $152.” darpa.mil ↩
7. Anthropic, Project Glasswing initial update (May 22, 2026): 23,019 vulnerabilities found across 1,000+ open-source projects; about 530 high- or critical-severity bugs disclosed to maintainers, 75 patched; “several maintainers have told us they’re currently severely capacity constrained, and some have even asked us to slow down our rate of our disclosures because they need more time to design patches.” anthropic.com ↩