Study Finds AI-generated Code Fails to Boost Productivity Due to Downstream Bottlenecks

A Harvard study covering 718 firms and drawing on the Jellyfish engineering analytics platform concludes that AI coding assistants and agents inflate code output without a matching rise in finished work, and that slower, heavier code review is a major reason. InfoQ reported the findings in an article by Sergio De Simone. The dataset spans roughly 300 million work events from about 700,000 workers across more than 700 firms, covering January 2021 through March 2026.
Data came from several systems: HR records, version control, issue management, scheduling tools such as Google Calendar, and generative AI tools including GitHub Copilot, Cursor and Claude Code. The researchers tracked three coding output measures — lines of code added, commits and pull requests — and used Jira issues and epics resolved per worker per month as their yardstick for completed engineering work, with issues standing in for individual units of work and epics for larger projects or features.
The two tool categories behaved very differently. Coding assistants produced small gains: 12% more lines of code, 9% more commits and 5% more pull requests, with only the commit effect statistically significant. Agents produced larger and significant increases across all three: 30% for lines of code, 20% for commits and 23% for pull requests. On completed work, assistants showed no increase in resolved issues, while agents delivered a 12% rise — well short of the 30% jump in lines of code.
To explain the gap, the authors looked beyond writing code to the rest of the pipeline: feature and scope planning, review, testing and deployment. They imputed time allocation from Jellyfish data and checked it against a survey of 100 engineers on Prolific, estimating that engineers spend 30-40% of their time writing code, with the rest going to these complementary tasks. When AI speeds up the writing stage, those other stages can become bottlenecks.
The review numbers are stark. Average pull request review time rose 49%, comments per PR rose 53%, and the share of PRs requiring changes doubled. The share of workers doing reviews increased 14%, and adopting AI to help with reviews did not improve matters. The authors also found the extra review effort could not be explained by larger individual PRs.
This is the crux for anyone evaluating AI coding tools. Lines of code and commit counts are easy to instrument, which makes them attractive as productivity proxies, but they measure activity rather than delivered value. A tool that triples drafting speed while doubling review churn can look impressive on a dashboard and neutral or negative in delivery terms. The study's use of resolved Jira issues is a more defensible outcome measure because it requires work to pass through review and testing, though it carries its own caveats: issue granularity varies by team, and resolution counts can be gamed or inflated by splitting work into smaller tickets.
The mechanism the authors describe is intuitive. AI-generated code arrives faster than humans can absorb it, and reviewers who did not write it must reconstruct intent before they can judge correctness, which is slower than reviewing a colleague's work built on shared context. The observed rise in comments per PR and in PRs needing changes is consistent with reviewers finding more to question, whether because of genuine defects, stylistic mismatch or unclear design choices. A likely trade-off is that teams gain raw throughput at the cost of a heavier human verification layer, shifting the constraint from typing to judgment.
Community reaction quoted in the article focused on code as a liability. One Reddit commenter noted that every line of code carries potential maintenance cost, and another pointed to the added risk of AI hallucinations, arguing agents can pursue a task in damaging ways without any intent. A third suggested mean time to recovery could become a more telling metric, reasoning that engineers who write and understand less of what they ship may face more frequent and harder-to-diagnose incidents as codebases grow. That is speculation rather than a measured result, but it points at a plausible second-order cost the study did not quantify.
Jellyfish itself sells a platform for measuring engineering performance, connecting tools such as GitHub and Jira to help companies understand developer productivity and software delivery. That makes the study's data source commercially relevant to the measurement market, and readers should weigh the usual caveats of vendor-sourced telemetry: tool coverage, sampling and how activity maps to real output. The headline findings nonetheless align with a widely reported pattern in which AI adoption shows up clearly in output volume and weakly in delivery metrics.
Why it matters: Engineering leaders buying AI coding tools should measure cycle time, review load and resolved work rather than code volume, since this study suggests gains at the writing stage can be absorbed by downstream review. Platform and tooling teams have a clear opening — AI-assisted review, smaller change sizes and better PR context are where the bottleneck now sits, and the finding that AI review assistance has not yet helped is an inference worth testing in your own pipeline. Product managers should expect agent rollouts to raise review capacity requirements before they raise shipped throughput.
Based on reporting from the original publisher. Visit the source for full context and later updates.
Publisher excerpt
A recent Harvard study of 718 firms, using the Jellyfish analytics platform, finds that although AI coding assistants and agents increase code volume, this does not translate into a corresponding increase in completed work, partly because AI-generated code increases the workload in later stages of development, particularly code review. By Sergio De Simone