Does better work always mean better workers?

A three-month randomized controlled trial with 133 practicing patent attorneys at eleven intellectual property law firms found that routine AI use improved drafting quality across the board, but its effect on professional judgment depended on seniority. The experiment, described in a National Bureau of Economic Research paper by David Autor and Tanya Rodchenko, randomized access to a then-unreleased Google Labs AI patent writing assistant, now part of Gemini Notebook. Two-thirds of lawyers at each firm received early access; the rest got some AI training but were withheld from the tool until after the study.

On drafting tasks, AI access raised scores graded by independent legal experts on five quality dimensions. At 10 days, scores rose by 0.34 standard deviations, equivalent to a 10-point percentile climb; by 90 days, 0.38 SD, an 11-point climb. Gains came from fewer poor scores and more good ones, with no change in excellent scores. Junior lawyers also saved time: 18 minutes faster than the control group's 124-minute average on the 10-day task.

At 90 days, lawyers completed a redlining task with AI prohibited, testing unassisted judgment. Those with prior AI access outperformed controls by 0.32 SD (a 9-point percentile climb), driven entirely by senior lawyers at 0.45 SD (13 points). Junior lawyers showed no average improvement; their scores bifurcated into more very low and more good scores, with no increase in excellent ones.

Qualitative analysis found junior submissions followed rigid formalism, copy-editing introductory sections and swapping synonyms rather than improving commercial scope, often diagnosing flaws without fixing them. This diagnose-without-execute pattern also appeared among unassisted control juniors. Senior lawyers spent longer, rebuilt claims from scratch, and described AI output as a logic auditor rather than a finished product.

Why it matters: For product teams and developers building AI tools for professional work, improving immediate output and building durable expertise are separate goals that may conflict for junior users. The authors note foundational expertise may be the missing link allowing AI-assisted repetition to translate into seasoned judgment, and warn that tools designed to produce better work today could hinder the development of tomorrow's experts.