Big news at the UN

Gary Marcus delivered framing remarks at the United Nations General Assembly Digital Cooperation Event, a session that also featured Yoshua Bengio and Nobel Laureate Maria Ressa, according to his account of the event.

In those remarks, Marcus said the current moment is one of moral, political and economic crisis and that choices made now around AI may reverberate for generations. He said he still considers a net positive outcome with AI possible, including in technology and medicine, but that the discourse around AI frightens him.

He rejected two positions: no regulation at all, and the claim that literal extinction is imminent. He said that scenario, taken literally, is extremely unlikely, and that scenarios for it underestimate human ingenuity and resourcefulness while many overestimate AI. What he said we face near term is instead wholesale deepfaked disinformation and unreliable but persistent AI systems stealing credentials and launching cyberattacks at scale. He also raised the possibility of a recession if bets on AI prove unsustainable, and said malicious actors should be expected to try to use current AI against financial systems and vital infrastructure such as trains or electrical systems.

Marcus called for increased reliability, better cybersecurity and genuine enforcement, possibly extending to product recalls and prosecution of companies that repeatedly put dangerous products on the market. He said AI agents with unrestricted internet access should be banned by governments until shown to be safe, and criticized OpenAI for releasing a new model harder to monitor than its predecessors, arguing monitoring is a foundation of cybersecurity and OpenAI should not have made that decision alone.

He listed three immediate steps he favors: an empowered international advisory commission to evaluate risks versus benefits of new systems before introduction; an international advisory commission to audit new systems on an ongoing basis, with authority to demand change or recall dangerous systems; and an international agreement not to deploy demonstrably harmful architectures.

Marcus also said that at almost the same time, more than 20 countries signed a call for actions around AI, apparently written in the last week, which he said captures much of what he has hoped for. He said many questions remain about how it will be realized, financed and governed.