Hot Takes

China's AI Labs Ran 24,000 Fake Accounts to Steal Claude's Brain

Riley Torres ·
China's AI Labs Ran 24,000 Fake Accounts to Steal Claude's Brain

Anthropic caught three Chinese AI companies running what it calls an "industrial-scale" operation to train their own models on Claude's answers. The numbers are specific enough to sting: roughly 24,000 fraudulent accounts and more than 16 million exchanges with Claude. The companies named: DeepSeek, MiniMax, Moonshot AI.

The technique has a polite name, distillation: a smaller model learns by watching what a bigger model produces and training itself to match the pattern. It doesn't steal the weights. It doesn't need to. Do it at volume and you get a model that replicates the teacher's reasoning style and phrasing habits without ever touching the original code. Crude by academic standards. Devastatingly effective in practice, especially for a company trying to close a capability gap without matching the teacher's compute bill.

Claude isn't officially available in China. Neither is ChatGPT. So the workaround for labs building frontier products there is exactly what Anthropic just described: spin up accounts somewhere accessible, query at industrial volume, collect the outputs, train on them. Twenty-four thousand accounts running over 16 million exchanges is not curiosity. It's a coordinated supply chain for someone else's intelligence.

DeepSeek is the name everyone already knows, having spent early 2025 as the efficiency story of the year after R1 reportedly matched frontier performance at a fraction of the claimed training cost. That efficiency claim was the whole pitch. If distillation on Claude's answers contributed to it, the comparison gets a lot less flattering. MiniMax and Moonshot are less famous outside China but are real, well-funded companies competing hard in a market where access to high-quality Western outputs is scarce and valuable. It's the same moat problem eating away at Western pricing power that showed up when Moonshot's own Kimi K3 spooked Wall Street a few weeks later, except this time the moat got copied instead of undercut in the open.

Here's the part that makes this genuinely awkward rather than simply satisfying: nobody's sure it's illegal. Anthropic's usage policy explicitly bans training competing models on Claude's outputs, and this is obviously that. But copyright protects expression, not reasoning patterns, and a Claude answer explaining how to structure a database query doesn't fit neatly into categories copyright law was built for. OpenAI is fighting the exact same question in the opposite direction right now. The legal framework has holes at every layer of this fight, which means Anthropic's real weapon here isn't a lawsuit. It's the disclosure itself: publish the receipts, make the extraction reputationally expensive, and let everyone watching draw their own conclusions.

The uncomfortable truth underneath all of it: distillation at industrial scale is operationally practical, full stop, this episode just proved it. Every AI company with a valuable model now has an adversarial monitoring problem, and the technical barrier to running your own version of this is exactly as high as getting API access through a friendly jurisdiction. Which is to say: not very.

Enjoyed this? Get more.

Weekly dispatches on AI culture, chatbots, and the robot future. No hype.

Free. Unsubscribe anytime.

#anthropic#claude#deepseek#china-ai#distillation