Import AI 459: AI oversight is difficult; scaling laws for protein folding models; and pricing the extinction risk of AI systems

Import AI 459 highlights four items.

A paper by economists with the University of Virginia, Anthropic, and the Bank of Canada estimates nominal AI GDP at approximately $250 billion in 2025, growing at roughly 2,600 percent per year in quality-adjusted real terms, while noting this growth is largely invisible in conventional GDP statistics. The authors report US compute spending rose from $37 billion in 2023 to $90 billion in 2024 to $219 billion in 2025, with US AI computing capacity growing more than 200 percent per year and quality-adjusted AI output rising roughly 2,290 percent in 2024 and 2,271 percent in 2025. They argue AI may differ from semiconductors and the internet because it may substitute for human labor, and recommend AI satellite accounts, better primary data, and incorporation of AI productive-capacity measures into medium-term projections. One author, Anton Korinek, is affiliated with Anthropic; the research was mostly done prior to him joining and outside his work at the company.

Researchers with the UK AI Security Institute published a paper arguing automated alignment research is harder than often assumed, citing optimization pressure toward human approval, alien mistakes, correlated research, research volume, and non-human-evaluable arguments. They suggest interventions across measurement, generalization, and scalable oversight, including red-teaming automated alignment programs.

Stanford University, Radical Numerics, the University of Michigan, and Salesforce Research released the Giant Permissive Image Corpus (GPIC), 100 million images with captions, all permissively licensed for research and commercial use, safety-filtered, deduplicated, and hosted on Hugging Face as 8,000 shards. GPIC includes 100M training, 200k validation, and 1M test examples, sourced from Flickr and Wikimedia under CC BY, CC0, Public Domain, and No-Known-Restrictions categories, with captions generated by Qwen3-VL-4B.

Biohub, founded by Priscilla Chan and Mark Zuckerberg, released ESMFold2, described as a world model of protein biology. The release includes ESMC, trained on approximately 2.8 billion sequences; ESMFold2; and ESM Atlas, covering 6.8 billion protein sequences and 1.1 billion predicted structures. Biohub reports designing binders against EGFR, PDGFRβ, PD-L1, CTLA-4, and CD45, with hit rates of 36–88% for compact minibinders and 15–29% for antibody-derived formats, and confirmed binding in laboratory experiments.