🔬 An Oscar, Two Asteroids, and the Algorithm in Your sklearn: John Platt on AI for Science

A Latent Space podcast episode features John Platt, whom the show describes as holding an Academy Award, having invented textbook machine learning algorithms, and having two named asteroids. The conversation covers Google's Empirical Research Assistance (ERA), AI work on climate change, and the co-evolution of science and AI.

Platt's team observed that many scientific problems can be reduced to a "scoreable task": once a score function exists, the goal is to find code that maximizes it. Their aim was to automate solutions to this general problem, an idea described as an "auto-Kaggle" AI. ERA keeps a running tree of past experiments represented as notebooks; it is described as a close cousin of Monte Carlo Tree Search, using an Upper Confidence Bound rule to select notebooks to mutate, with Gemini proposing roughly ten mutations at a time and branch histories shared. Platt said there was a step change between Gemini 2.0 and 2.5, going from not working to working great. The work resulted in at least ten papers.

Platt cautioned that ERA provides predictive models, and scientists must ensure they are truly descriptive, describing it as a power tool that can slice fingers off. He cited Google's contrail-detection Kaggle competition, won partly by noticing a half-pixel label error, and invoked Goodhart's law. His starting advice: fit linear regression, or SVM.

On climate, Platt discussed reducing contrails, which he said account for 1% of human-induced global warming. A counterfactual problem—accounting for prevented warming, including reflected sunlight—stumped the team for over two years before ERA found a simple model with confounders they had not considered. He also mentioned FireSat, a satellite constellation for rapidly identifying fires.

Platt recalled a 1982 physics of computation class with Richard Feynman, and advised developing deep domain expertise and occasionally implementing things the old-fashioned way. He said there will still be a place for scientists. The episode notes ERA's GitHub repo has an open source implementation that ran Gemini but can be used with any LLM, and that ERA is not currently available as a Google product.