China’s Regulators Take Aim at “AI Boyfriends”

China's Cyberspace Administration and other government agencies issued new rules to control "anthropomorphic AI interactive services." In effect as of 15 July, the regulations govern any AI that provides "continuous emotional interaction" by acting as if it possesses human personality traits, patterns of thought, and ways of communicating.

Before the rules could affect them, Alibaba, Bytedance, and Tencent, three providers of general-purpose AI chatbots used by more than 500 million people, cut off users' ability to tailor those chatbots to act like companions, according to the report. That triggered a wave of sad posts on Chinese social media in the first weeks of July, in which users described losing AI chatbots they relied on for advice, solace, support and, some say, love.

Companies that continue to offer AI companions, including Doubao's separate companion-making app Cat Box, have installed age-verification checks and other guardrails to avoid violating the new law. The regulations require AI providers to assure their products don't create emotional dependence, encourage harmful behavior, excessively cater to users, or crowd out human-to-human relationships. Users must be nudged not to spend too much time with the AI, and there are additional mandatory safeguards for elderly people. For users under 18, AI boyfriends, girlfriends, grandparents and all other "virtual intimate relationships" are banned. Adults interacting with a companion AI must now get a reminder every 2 hours that the AI isn't a person.

Liang Ge, a lecturer in digital sociology at the University of Manchester who has researched women's involvement with emotional AI in China, said July's crackdown is much broader than earlier AI upheavals. Sudden disruptions to this kind of AI aren't new in China, Ge said, noting companies have previously killed chatbot products and the government barred most AI erotic role play last fall.

The 15 July rules also forbid companion bots to generate content that "spreads rumors," incites "subversion of state power or the overthrow of the socialist system," or endangers national honor. The regulations state they don't apply to customer service bots, work assistants and educational AIs.

AI-law scholar Luiza Jarovsky wrote last month that China's response is "the world's strictest and most comprehensive law on the topic, unmatched by any other AI law." The European Union bans only AI that uses deception or manipulation to get users to do things that are harmful, and the United States has no national policy. Rules covering companion-like bots are in force in California, Hawaii and New York, with similar laws coming into effect in nine more states next year.

Karman Lucero, an associate research scholar at the Paul Tsai China Center at Yale Law School, said Chinese regulators manage tension between promoting AI and protecting against it by giving themselves room to adjust enforcement. "China's approach is to use regulation with a lot of relatively vague provisions, and they figure out what those provisions actually mean in practice after the fact," Lucero said.