AivexaNewsSearch
AI news for builders and product teamsChecked every hour

AI coding agents still need human cleanup on nearly three out of four tasks in a real-world test

Collected Nov 8, 2026

Alibaba has introduced Co-Design Bench, a coding-agent benchmark assembled from real user requests that had already tripped up AI agents in practice. On it, the strongest performer, GPT-6 Astra, fully solves just 26.4 percent of tasks, meaning human cleanup is still needed on nearly three out of four. The same models reportedly score above 99 percent on older benchmarks.

The gap is the story. Benchmarks built from curated, self-contained problems tend to reward pattern matching against well-specified inputs. Co-Design Bench instead draws from messy real-world requests, which is where agents apparently struggle. This suggests older scores largely measure performance on cleaned-up problems rather than the ambiguous, context-heavy work developers actually hand off.

Why it matters: teams evaluating coding agents should treat headline benchmark numbers with skepticism and test against their own real requests. For developers, the practical takeaway is that agent output still needs review and repair, so workflows should keep a human in the loop rather than assuming near-complete automation.

Read at The Decoder

Based on reporting from the original publisher. Visit the source for full context and later updates.

Publisher excerpt

Alibaba's new coding-agent benchmark, Co-Design Bench, tests AI agents on real user requests that already tripped them up in practice. Even GPT-6 Astra, the top performer, fully solves only 26.4 percent of tasks. On older benchmarks, the same models score above 99 percent. The article AI coding agents still need human cleanup on nearly three out of four tasks in a real-world test appeared first on The Decoder .