Anthropic trains Claude to mimic safety classifier without consulting policies
Anthropic trains Claude to mimic safety classifier without consulting policies
Claude now silently checks whether its responses may have contributed to harm, while Anthropic distills an expensive safety classifier into a cheaper self-assessment mechanism.
Claude now quietly performs an internal check every time it responds to a user, trying to determine whether its answer may have contributed to harm. The results are sent to Anthropic specialists for further review.
Anthropic’s most accurate tool for detecting abuse is a classifier built on Claude Mythos Preview. Before making a decision, it consults all of the company’s safety policies. However, this procedure is too computationally expensive to run for every conversation.
Anthropic therefore trained Claude to reproduce the classifier’s decisions without consulting the safety policies themselves or repeating the classifier’s reasoning.
When checking requests to Fable 5, the new mechanism detected 78% of the cases flagged by the original classifier. Among high-priority cases, the detection rate reached 94%, while the false-positive rate was only 1%.
Why it matters
AnalysisIt shows how AI labs try to scale safety monitoring without paying the full compute cost of policy-level review. For users, it means more automated risk checks built into model responses.
Discuss in community
Share your questions and insights with developers
Related in Ecosystem
Practical guides, core terminology, and agent skills
Experimental Claude AI Model Found Vulnerabilities in Chrome, Safari, Android, Windows and Linux
Anthropic published results of testing its experimental Claude AI model, including 62 Chrome vulnerabilities, a Safari exploit and a 53-minute Linux kernel root exploit.
Anthropic removes five-hour Claude usage limits for Ultra subscribers
Anthropic is dropping five-hour usage limits for Claude, but only for verified Ultra subscribers, while renaming its Max tiers without changing prices.
Computer Use via AI
Anthropic's technology (introduced in Claude 3.5 Sonnet) enables neural networks to control operating systems like a human: taking screenshots, analyzing window coordinates, moving the mouse cursor, clicking buttons, and entering text without using specialized APIs.
Red Teaming in AI
A methodology for aggressive stress-testing AI models by specialized ethical research teams (Red Teams). Experts simulate malicious actions, identifying security gaps, jailbreaks, and prohibited responses before the model reaches millions of users.