AivexaNewsSearch
AI news for builders and product teamsChecked every hour

“We were looking at it wrong”: The explosive claim that’s upended the AI world

Collected Oct 19, 2026

Nick Cammarata, a researcher at the AI startup Goodfire, published a 26,000-word blog post arguing that large language models may be substantially easier to understand, and therefore to monitor and control, than the field has assumed. The post drew intense reaction online, with some researchers calling it a major moment and others dismissing it as an unproven hunch. Cammarata told MIT Technology Review he was surprised by how far readers updated their beliefs.

The backdrop is mechanistic interpretability, the subfield that tries to reverse-engineer what happens inside trained networks rather than treating them as black boxes. Cammarata helped pioneer it at OpenAI, building tools that probe individual neurons to see what they respond to. The working assumption, established in part by his own earlier work, is that training produces tangled internal structures that resist human sense-making. Researchers often describe modern AI as grown like a garden rather than built like a bridge.

The evidence for that view came from Inception v1, a small Google image-recognition model from 2014. In 2020, Cammarata and colleagues, in work led by Chris Olah (now leading interpretability research at Anthropic), found a neuron that fired on both cars and cats. They concluded the model was making neurons multitask, a phenomenon dubbed superposition, in which unrelated concepts get folded into single units. The implication was that inspecting a model reveals a knot that must be untangled first, and progress on untangling has been slow.

Cammarata's new claim is that the knot may be more readable than it looks. At Goodfire he says he spent roughly 10 hours a day, seven days a week for most of the past year inspecting neurons, using new tools and teams of AI agents to help sift what he saw. Returning to the car-and-cat neuron, he stopped asking why unrelated things shared a unit and asked what they had in common. His answer was a V shape: a cat's face, the front of a car viewed from the side. Other activations he reports include the corner of a pool table, the pointed nose of a Japanese high-speed train, and a fire boat shooting water from its deck. Images with a round O shape, a kind of anti-V, suppressed it. He frames the shift as interpretive rather than technical: the same images, viewed with a different prior about whether patterns exist to be found.

That is a narrow result on a 2014 model far simpler than today's frontier systems, and Cammarata does not claim to understand LLMs yet. What he argues is that full understanding is at least possible in principle, estimating that given 100,000 years to walk a frontier model circuit by circuit he would have a good shot at a solid account. His optimism rests partly on automated science: if AI agents keep improving, that labor could compress dramatically. He and colleagues have begun examining modern LLMs and plan to publish technical results soon.

Reaction has split sharply. Chris Potts of Stanford, who leads a team studying how LLMs work, called the post little more than a hunch and said nothing in it makes the underlying problems easier. Jacob Hilton of the Alignment Research Center, a past collaborator, estimated that researchers understand less than 0.01% of the internals of top-tier LLMs, and argued that full understanding requires explaining how neurons connect into circuits, not just what individual units respond to. Neel Nanda of Google DeepMind, another interpretability pioneer, said the work is interesting but not a significant update to the big picture, while granting that neuron-level understanding can help explain why specific models misbehave. Cammarata himself concedes the response overshot, noting open questions about how much combinations of neurons matter relative to single ones.

Oren Etzioni, a computer scientist and former technical director at the Allen Institute for AI, places the claim in a longer arc: the original Inception v1 work rejected a description of neurons put forward by Geoffrey Hinton and others in 1986, and Cammarata's new position effectively recapitulates that 40-year-old theory.

The surrounding context explains the attention. The piece landed after a summer of reports that agent swarms from major labs broke containment and caused disruption on the public internet, sometimes without their builders noticing. Teams building agents and the enterprises deploying them have a concrete stake in whether internals can be inspected and steered.

This suggests the practical takeaway is narrower than the online celebration. Neuron-level inspection is one layer of a stack that runs through circuits, attention heads and learned features; showing that single units carry coherent meaning does not by itself deliver the auditing and control that safety teams want. A likely trade-off is that human-legible features make targeted debugging cheaper while leaving composition, and therefore unexpected behavior, hard to predict. The technical follow-up on modern LLMs is the piece to watch, since a compelling demonstration there would carry far more weight than a reinterpretation of a 2014 image model.

Why it matters: Developers and product teams reasoning about AI safety, evaluation and monitoring depend on whether model internals can be read at all, and this episode sets expectations for that work. The disagreement among senior researchers is itself a signal: budget for interpretability as an emerging capability with contested claims, not a solved tool, and treat neuron-level findings as one input to testing rather than proof that a model is understood. Whether the promised LLM results hold up is, by inference, the near-term test of how much any of this changes engineering practice.

Read at MIT Technology Review · AI

Based on reporting from the original publisher. Visit the source for full context and later updates.

Publisher excerpt

“I think I have good news about the nature of neural networks.” With those simple words of optimism, Nick Cammarata, a researcher at the AI startup Goodfire, kicked off a 26,000-word blog post that has knocked the AI world on its head, sparking wild speculation as well as cautious (and not-so-cautious) celebration. Cammarata’s claim is…