OpenAI says a distillation campaign got its models to decrypt their own protected reasoning

OpenAI's account is that the cryptography held. "The operators did not break our encryption, compromise a database, or gain direct access to stored user conversations," the company wrote in an unsigned blog post ^[4]. "Instead, they manipulated model interactions so that protected reasoning could be reproduced in forms visible to the requester in a coordinated, scaled manner that violated our terms of service." ^[5] The operators copied encrypted reasoning out of one conversation, then asked the model in a separate conversation to decrypt it and write it out in plain text ^[12]. OpenAI called the method "novel" ^[14].

The fix shows where the check was missing. OpenAI says it closed a bug that let users take encrypted data from one conversation and decrypt it in another ^[11]. Until then, the reasoning stayed encrypted wherever the operators carried it, and a second session would still turn it back into text when asked ^[12]. Encrypting the output protected it here only as far as the service refused to decrypt it for a conversation that did not produce it ^[12][11]. OpenAI says the same vulnerability exists in other AI models, and it has shared details of the incident with groups including the Frontier Model Forum ^[9][10].

The traffic was spread thin. At the July 24-25 peak, matching prompts averaged four per user ^[19], so no single account had to look heavy. Suspicious users then grew 3.75-fold in three to four days ^[21]. First sighting to full disruption took 27 days ^[20]. That spread fits the pattern Google and other security firms describe in Chinese distillation work: thousands of individual accounts bought on black or gray markets for models like ChatGPT and Claude, then used to flood those models with millions of prompts ^[8].

CyberScoop reported that the post cites no technical evidence or reasoning for naming Moonshot AI ^[15]. OpenAI told the outlet it was withholding further detail "for security reasons" ^[16]. CyberScoop said it had asked Moonshot for comment ^[18]. The technique and the counts come from OpenAI's post; the Moonshot link is OpenAI's assertion. American AI companies and the U.S. government have previously accused Chinese firms, Moonshot among them, of "systematic" distillation attacks on their latest models ^[7].