AivexaNewsSearch
AI news for builders and product teamsChecked every hour

Learning the language of machine learning

Collected Nov 17, 2026

Jacob Andreas, an associate professor in MIT's Department of Electrical Engineering and Computer Science and a member of the Computer Science and Artificial Intelligence Laboratory, has been granted tenure. The profile of his work centers on a single ambition: rather than reverse-engineering a trained network node by node with outside tooling, train models to describe what they have learned in human-readable language, accurately. He frames the shift in debugging practice directly — where researchers once inspected parameters to judge whether a system made sense, the scale of modern neural computation now demands what he calls "mind-reading tools" that translate that computation into terms people can follow.

Andreas's path into the field ran through language. Raised in the San Francisco Bay Area, he read widely as a child, and a book from a family friend featuring a character who hacks a computer system to save the day first drew him toward computing. A BASIC Engine his father brought home followed, and he began programming. He majored in computer science at Columbia University, unsure whether he wanted research, engineering or a startup; friends joining faculty labs prompted him to email essentially everyone in the department working on anything AI-adjacent, and the natural language processing faculty answered first. That landed him in the lab of Michael Collins, the Vikram S. Pandit Professor of Computer Science, where his undergraduate thesis adapted statistical techniques for representing linguistic meaning into stronger machine translation systems.

He went on to graduate study at the University of California at Berkeley, after a year earning a master's degree at Cambridge University, and joined the lab of Berkeley computer science professor Dan Klein. His timing put him inside a structural change in NLP: the field moved from statistical methods, which use transparent mathematical formulas to analyze text, to deep learning, whose opaque algorithms learn complex meanings automatically from very large text corpora. He describes the period as an enjoyable one in which the modern toolkit was still being invented. Collaborations with computer vision researchers produced systems that generated natural language well by being taught to "see" their surroundings; neural networks, he recalls, finally gave language and vision work a shared representational vocabulary. His thesis covered neural module networks — small network fragments handling specific jobs such as localizing objects or logical reasoning inside a larger system — but the more speculative threads alongside it, interpretability and generating natural language descriptions of a model's internal states, became the agenda for his MIT group.

After a year at the startup Semantic Machines, which spun out of his Berkeley lab, Andreas joined MIT in 2019. His group now works on two fronts. The first is a science of large language models: understanding the algorithms they deploy as they learn and process information, and building tools that help researchers predict how a trained model will behave in the real world. One family of techniques trains LLMs to explain what is happening inside other neural networks in ordinary language, and can attach a natural language description to every information-processing node in a very large network — one node might be labeled as detecting "patches of grass near animals" in input images. Andreas treats that labeling capability as a starting point for higher-level questions about how models represent meaning.

The second front is training methodology: new algorithms for training models, and methods for predicting how a particular dataset or individual example will affect specific aspects of model behavior. Here he is candid about the gap — the field lacks a good scientific theory of what happens when a model is trained on a given dataset, which he calls a major current challenge.

This pairing is worth understanding on its own terms. Interpretability tools generally fall into two camps: mechanistic approaches that decompose internal activations, and behavioral testing that probes outputs. Using a language model as the translator for another network's internals is a hybrid — it borrows the fluency of one model to narrate the computation of another. A likely trade-off is that the explanation is only as faithful as the narrator: a fluent description can sound convincing while misrepresenting the underlying circuit, so such labels need independent validation before anyone relies on them. The node-level labeling example also hints at practicality — attaching a sentence to every node in a massive network produces enormous output, so the useful signal is probably aggregate structure rather than a per-node catalog.

Andreas also co-teaches 6.461 and 6.861, the undergraduate and graduate natural language processing courses, which he and colleagues rework annually to track the field. He puts the pace of change at a full turnover every three to five years. Teaching carries its own pressure now that a large language model can complete many assignments; his answer is that a deep foundation in how the technology works underneath is what distinguishes students when everyone has a coding agent available.

Looking ahead, he expects interpretability to matter more, not less, for making agents safe enough to deploy in high-stakes situations, and especially for deep-learning models used in scientific discovery that are trained on raw lab data no scientist has yet characterized. Getting such models to articulate hidden knowledge about phenomena nobody has written down explicitly is, in his view, both the field's coming challenge and its opportunity. Outside the lab he backpacks at least once or twice each summer and is working through the 4,000-foot peaks of New Hampshire's White Mountains with a colleague from the mathematics department; with tenure secured, he plans to return to chamber choir singing, a pursuit kindled by chapel choirs and bell ringing at Cambridge.

Why it matters: Teams deploying models in regulated or safety-critical settings increasingly need to justify behavior, not just measure accuracy, and node-level language descriptions point toward audits that read like documentation rather than activation dumps — though the faithfulness question above is the open risk. For product teams, dataset-to-behavior prediction is the more immediately practical thread, since it speaks to data curation decisions made before training. Andreas's teaching stance is a signal too: as coding agents absorb routine implementation, the durable differentiator for engineers is understanding what happens under the hood.

Read at MIT News · AI

Based on reporting from the original publisher. Visit the source for full context and later updates.

Publisher excerpt

By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.