The Shape of AI: Jaggedness, Bottlenecks and Salients

The essay revisits the "Jagged Frontier" term the author and co-authors introduced in 2023 to describe AI doing some work very well and other work very badly in ways that do not map to human intuition about task difficulty. The author argues jaggedness will remain a major feature of AI, disagreeing with Tomas Pueyo's viral X post, which held that the growing AI frontier will outpace jaggedness since human ability is mostly fixed.

The author points to LLM memory as a persistent source of jaggedness: LLMs do not permanently remember new tasks and learn from them, and this may be harder to solve than researchers expect. A group of scientists recently mapped the shape of AI ability and found it growing unevenly, with reading, math, general knowledge and reasoning improving rapidly while memory showed little improvement. Better prompting or models, including GPT-5.2 versus GPT-5, might change the frontier's shape but not remove jaggedness.

Bottlenecks arise when a system is only as functional as its worst components. Cited AI weaknesses include vision systems that cannot read medical imaging well enough to replace doctors, models too helpful when they should push back to replace therapists, and hallucinations that persist, ruling out tasks requiring 100% accuracy. Other bottlenecks are unrelated to ability: drug candidates may be identified faster, but clinical trials still need human patients and the FDA still requires human review.

A study found GPT-4.1, properly prompted, reproduced and updated an entire issue of Cochrane reviews (n=12) in two days, about 12 work-years of traditional review, screening over 146,000 citations and outperforming human reviewers on accuracy. But it could not access supplementary files or email authors for unpublished data, under 1% of errors that still prevent full automation.

The author cites historian Thomas Hughes's "reverse salients," single problems holding back a system. Google's Nano Banana Pro, combining image creation with a smart directing model, is presented as removing image generation as a bottleneck for presentations.