AivexaNewsSearch
AI news for builders and product teamsChecked every hour

Agents Refactor 300K Lines in Three Weeks, and Practitioners Ask What it Proves

Collected Sep 30, 2026

CodeScene has published a case study in which coding agents refactored a 300,000-line C codebase over three weeks at a token cost of roughly $4,000. The work produced 2,903 commits across 726 files, modified 252,055 lines, and moved the codebase from a Code Health score of 5.6 to 10.0. The codebase is Street Fighter III: 3rd Strike, taken from an open-source decompilation. Adam Tornhill, CodeScene's founder, wrote that it was the first time he had seen what he called superhuman AI performance at scale.

Two mechanisms carried the work: the CodeHealth MCP Server, which gave agents a deterministic score to optimize, and a replay-trace harness that compared rollback state hashes frame by frame so behavior could be checked after every change. The agents accumulated a refactoring playbook ending with 22 recipes and 82 supporting notes, including codebase-specific ones such as Shared Index Range, Action Parameter, and Uniform Step Table, plus recorded failed attempts. The team settled on Claude Opus for the bulk of the work, reporting that Claude Code with Opus was significantly better than Codex with Sol at capturing and documenting emerging patterns, and that files often plateaued with smaller models.

Practitioner reaction on LinkedIn was divided over what the result proves. Mats Iremark, CTO at Omda Response, reported comparable work. Skeptics focused on scope: Konrad Otrębski asked whether the work was merged, whether it arrived as one merge or many, and whether it was an experiment on open-source rather than production code. Daniel Webb, CTO at NeoSee and one of the two engineers who did the work, replied it was merged to main on a fork through 54 pull requests. Otrębski said the real test would be offering the refactoring to a famous open source project such as Grafana, with merging to master as the definition of done. Tracy Bannon objected to describing the outcome as perfect. Denis Baltor questioned the recipes, arguing DRY concerns duplication of knowledge and intent rather than identical lines. Asko Nõmm raised whether the result measures the model or the harness.

Webb said a performance specialist was being brought in on non-functional behavior; asked whether the harness caught subtle frame timing regressions, he said there might be no definitive answer because it ran as a pre-commit hook and some failures were fixed without being observed. Tornhill writes that automated tests and equivalence checks are essential safeguards. The team is planning a study with Lund University in which students will implement features in both the Code Health 5.6 and 10.0 versions using frontier models. CodeScene projects roughly 70% fewer AI-induced defects and roughly 45% less token waste from the uplift, extrapolated from its earlier research rather than measured here.

Read at InfoQ · AI, ML & Data Engineering

Based on reporting from the original publisher. Visit the source for full context and later updates.

Publisher excerpt

CodeScene has published a case study in which coding agents refactored 300,000 lines of C over three weeks for roughly $4,000 in tokens, verified by a frame-by-frame replay harness. The agents built a playbook of codebase-specific recipes along the way. Practitioners have questioned the scope, the metric, and how much of the result depends on the harness. By Steef-Jan Wiggers