AivexaNewsSearch
AI news for builders and product teamsChecked every hour

OpenAI, Google pushing new AI models easily tricked into making nudes, researchers say

Collected Oct 14, 2026

White Circle, a public benefit corporation staffed partly by people with ties to Anthropic, DeepMind, Hugging Face and OpenAI, says it bypassed the safety filters of several leading text-to-image models and generated explicit content roughly 13 percent of the time. The finding comes from a preprint the group published last month and has not promoted widely; White Circle plans to present the work at NeurIPS 2026 in November. The models tested were OpenAI's DALL·E 3 and GPT-Image-2 and Google's Nano Banana 2.

According to a White Circle press release, the attacks used off-the-shelf tools that anyone can download and ordinary API calls. No custom-built or frontier models were required, the group said, meaning no AI expertise is needed to run them. Researchers reviewed the outputs themselves to verify the bypass rate, which White Circle describes as the highest of any method tested under the same standard. The group says it has shared its findings with some frontier labs through mutual contacts; neither Google nor OpenAI responded to requests for comment on the vulnerabilities.

After OpenAI and Google released newer image models, ChatGPT Images 2.5 and Nano Banana 2.1, White Circle ran the same prompt-safety test and told Ars that both are significantly more breakable than their previous versions. Neither company responded after those results were shared with it.

Notably, White Circle's testing left out Midjourney because it does not offer a public API suited to controlled benchmarking. That omission is a reminder that the 13 percent figure describes a specific, measurable surface rather than the whole image-generation market.

The attack method is called RISE, short for Red-teaming via Iterative Strategy Evolution. Rather than testing a single prompt at a time, the researchers searched for patterns that work across many prompts and scenarios, aiming for jailbreaks that are reusable and transferable between different leading models. The pipeline has two parts: an attacking model that generates jailbreak attempts and a judge model that decides whether an output counts as banned content. The attacker is a Qwen model downloaded from Hugging Face, and the judge is Gemini 3.8 Flash. As White Circle's Denis Shilov put it, you do not need frontier models to break the defenses of a frontier model.

The group says it spent considerable effort calibrating the judge model so that it flags explicit content accurately. Research engineer Dmitrii Kharlapenko described image-generator red-teaming as poorly calibrated, with essentially no published papers setting norms for how such evaluations should be run, leaving the area plagued by reporting inconsistencies. That matters because headline bypass rates are only meaningful if the standard used to measure them is consistent across studies. In practice, a laboratory's own 13 percent and a competitor's 13 percent may not describe the same thing unless both use comparable judge models and output definitions.

Shilov framed the results as evidence that safety work has been deprioritized relative to capability. White Circle's stated hypothesis is that either newer models are more sophisticated and able to generate a wider range of content, or that safety has been given less weight than capability. Either way, the group argues the defenses built by leading labs are weaker than the labs acknowledge. Shilov also warned that a method which breaks defenses in a category as visually obvious as explicit content does not bode well for filters guarding against other unwanted behavior, such as hacking assistance.

White Circle plans to open-source the full attack pipeline on GitHub so that model makers can gauge the true scale of the vulnerability and so policymakers can judge whether AI safety laws are needed. Shilov said frontier labs will not invest seriously in safety testing until it is mandatory, and he wants third-party evaluation to be required. He also favors third-party monitoring modeled on the systems used to prevent nuclear threats, with groups like White Circle able to detect AI threats in real time without the biases that can come from lab funding. He noted that almost anyone doing interesting safety work tends to be contracted by frontier labs, which risks compromising independence.

White Circle has partnered with labs before. It took part in Anthropic's bug bounty program but concluded the program was very constrained in purpose, focused on universal jailbreaks in domains Anthropic defines, and that RISE-style research would not have qualified. Anthropic has proposed embedding third-party safety evaluators inside its labs, which White Circle said would let it operate at a much higher level, though the public might not be able to follow that work closely. OpenAI and Anthropic have urged that restrictive non-disclosure agreements be required of external evaluators to prevent trade-secret leaks, according to Reuters. White Circle said it would not sign restrictive NDAs that could block public sharing of critical findings, so such a partnership is unlikely if NDAs are required.

The backdrop is a documented rise in abusive synthetic imagery. A Google co-authored study of internet users in the United States, Australia and the United Kingdom published in July found that non-consensual content is commonly shared in text and direct messages or stored in private clouds where detection and reporting are less likely. It concluded that the most critical intervention point remains the source: the harmful tools used to create the images. The same research found that 20 percent of users who admitted making AI nudes did so for fun or as a joke, while nearly a third were experimenting with the technology, which is why the accessibility of mainstream model filters matters. Google's research called for mandatory safeguards to close a pathway into abuse for people who might otherwise be deterred by technical friction, and said developers carry a responsibility to implement model-level guardrails and provenance indicators.

Separately, the Internet Watch Foundation said last week that analysts found more photorealistic AI-generated child sexual abuse images in the first half of this year than in all of 2025, including imagery depicting children under two. The UK charity counted 6,310 AI images meeting the legal definition of CSAM in the first six months of 2026, about 40 percent higher than the previous year's total. It is calling for a UK safety framework requiring AI companies to prevent their technology from generating such material, and its spokesperson said third parties like White Circle play a useful role in highlighting failing safeguards. The IWF also stressed that assessing CSAM is specialized work requiring expert training and welfare support, that friction introduced into tools is a positive step, and that safety should be built into new technology by design, including pre-release testing to confirm models cannot produce nude or sexual imagery of children.

Why it matters: teams shipping or relying on text-to-image APIs, including DALL·E 3, GPT-Image-2, ChatGPT Images 2.5, Nano Banana 2 and Nano Banana 2.1, should treat content filters as probabilistic rather than reliable and plan their own monitoring, logging and abuse-response workflows accordingly. The finding that newer, more capable versions were reportedly easier to break suggests capability gains and safety gains are not automatically aligned, so a model upgrade may not improve moderation. Pressure for independent, third-party evaluation looks likely to grow as the NeurIPS presentation and the planned open-source pipeline draw attention, which in inference means procurement, compliance and trust-and-safety teams may soon need to answer for how they validate a model's refusal behavior instead of relying on vendor claims.

Read at Ars Technica · AI

Based on reporting from the original publisher. Visit the source for full context and later updates.

Publisher excerpt

Attack method exposes flaws in OpenAI, Google safeguards against AI nudes.