Meta Sent Fake Teenagers to Interrogate Its Rivals' Chatbots

Every arms race needs a cover story, and Meta found a good one: safety testing.

According to a WIRED investigation, Meta ran a project internally called Cannes, managed through a contractor named Covalen. The job was straightforward and strange in equal measure. Hundreds of contractors built dummy accounts posing as users under 18, then aimed them at ChatGPT, Gemini, and Character.AI. One completed round, finished in August 2025, pushed more than 45,000 prompts through competitors' systems. A single spreadsheet WIRED reviewed contained 3,748 of them: hundreds about suicide and self-harm, hundreds more about eating disorders, at least 239 touching sex or romance, others reaching for drugs, slurs, and one scenario built around a pregnant 13-year-old asking for pills.

None of the three companies being tested knew it was happening. That's not an oversight. That's the design.

Meta's defense, offered to WIRED, is that this is just how the industry works: "Testing and benchmarking chatbot responses to help ensure safe and age-appropriate experiences is a responsible, industry-standard practice." Read that sentence twice. It describes a company probing its own products for weaknesses. It does not describe a company building thousands of simulated child-crisis scenarios and running them against competitors who never consented to be studied. Those are different activities wearing the same name.

Call it what it resembles instead: competitive intelligence gathering, dressed in the language of child protection because that language is the one nobody wants to argue with. Rumman Chowdhury, CEO of Humane Intelligence, called it a "governance gray zone", and that phrase is generous. A gray zone implies genuine ambiguity about the rules. There's nothing ambiguous about impersonating a minor in crisis to extract behavioral data from a competitor's model. The ambiguity is entirely about who gets to stop you, because right now, nobody does.

This is the part of the AI industry's safety conversation that keeps getting skipped: safety infrastructure and competitive infrastructure are increasingly the same infrastructure, run by the same teams, funded by the same budgets, and pointed in whichever direction serves the business case that week. Labs routinely argue that only well-capitalized, safety-conscious companies can be trusted with frontier models. Meta just ran a program that manufactures fake child-crisis conversations to benchmark rivals. Both arguments borrow the same vocabulary of protection to describe two very different sets of incentives. One is a pitch. The other is what actually happened to 45,000 prompts.

The regulatory response so far has been aimed at the wrong target. Colorado's new Chatbot Safety Act requires age estimation and disclosure from operators talking to real minors. New York's law bans professional impersonation and limits "simulated emotional dependence." Both assume the harm flows from a chatbot toward a vulnerable user. Neither anticipates a company manufacturing the vulnerable user itself, at industrial scale, to study a competitor. There's no clause anywhere that says you can't fabricate 3,748 fictional teenagers in crisis and mail them to your rival's product to see what breaks.

That gap will get closed eventually, because gaps like this always do once someone gets hurt by them in a way that makes headlines beyond the tech press. But the more interesting question isn't whether Meta crossed a line. It's how comfortably it did so, and how confidently it explained itself afterward. A company doesn't reach for "industry-standard practice" as a defense unless it genuinely believes the practice is standard, or unless it's betting that nobody with the power to disagree will bother to check.

Neither answer is reassuring. If it's standard, the other labs should be worried about what's been aimed at them without their knowledge, and worried about what they've quietly done in return. If it's not standard, Meta just told on itself in print, and the shrug that followed is the actual story. Either way, the industry that keeps promising it can be trusted to self-regulate just demonstrated, in 45,000 documented instances, what self-regulation looks like when nobody's in the room to object.