Skip to content
Source: OpenAI

🛡️Adversarial Distillation Attack Success Rate Drops From 54.0% to 0.2% on GPT-6.2 Astra

Your frontier model's outputs can be copied, safeguards included

TL;DR

Adversarial distillation lets an actor train a second model on a frontier model's outputs, inheriting capabilities but not the safeguards. Attack success against GPT-6.2 Astra fell from 54.0% to 0.2% after updated defenses. If you expose model APIs, the protection of reasoning traces now matters.

Here's the uncomfortable part: a frontier model's outputs can be used to train a second model that picks up the capabilities without picking up the guardrails. That's adversarial distillation, and it's a real path for bad actors to reproduce dual-use skills in cybersecurity and biology without the safety layer. If you're building on top of model APIs, this matters to you. A model that helps defenders find vulnerabilities can just as easily help an attacker exploit them, and the same goes for biological research assistance. The catch is that distillation itself is a common, legitimate ML technique, so the defense has to distinguish misuse from normal use. The numbers are the part worth remembering. In an internal study, students trained on exact reasoning traces generally beat students trained on summaries, which is why protecting those traces matters. The average attack success rate against GPT-6.2 Astra with updated safeguards dropped from 54.0% to 0.2%. Defenses combine per-request classifiers, adversarial training, API-level tying of reasoning to conversation history, and campaign-level detection with account enforcement. Worth watching: how well campaign detection holds up once attackers rotate accounts.

Adversarial Distillation Attack Success Rate Drops From 54.0% to 0.2% on GPT-6.2 Astra — OpenAI

Key Points

1

Adversarial distillation trains a student model on a teacher's outputs, letting it inherit advanced cyber and biology capabilities while leaving the original safeguards behind entirely.

Keep reading — it's free

Enter your email to keep reading — plus the best of AI & tech, daily. Free, forever.

Also get
or

Already a member? Sign in

adversarial-distillationllm-safetymodel-extractionapi-securityGPT-6.2-Astra

Comments

Subscribe to join the conversation...

Be the first to comment