Skip to content

Anthropic's Constitutional Classifiers: A Jailbreak Defense That Got 40× Cheaper

Constitutional Classifiers is Anthropic's production defense against universal jailbreaks in Claude — prompting strategies that reliably bypass a model's safety training across nearly the entire space of queries in a harmful domain, turning the model's own capability into a scalable uplift tool for people who couldn't otherwise get that help. Anthropic's stated worst-case concern is CBRN (chemical, biological, radiological, nuclear) weapons uplift: a jailbreak that works universally doesn't need to fool a human reviewer once, it needs to work reliably enough to be a repeatable resource. The first version shipped and was documented in February 2025; a second, production-hardened version — informally "Constitutional Classifiers++" — followed after real deployment experience on live traffic. Together the two versions form one of the most thoroughly quantified case studies available of how a safety guardrail evolves once it has to survive contact with production cost and latency budgets, not just a benchmark.

The problem

Safety behavior baked into a model's weights via RLHF is hard to iterate on independently — every adjustment risks retraining dynamics elsewhere in the model, and there's no way to measure or patch the safety layer without touching the model itself. Universal jailbreaks make this worse: a single successful bypass technique, once discovered, tends to generalize across an entire harmful category rather than failing on the next similar prompt. Anthropic needed a way to defend against that class of attack that could be measured, iterated, and cost-optimized on its own, independent of the base model's training cycle — while not making Claude meaningfully less usable for the overwhelming majority of harmless requests.

What they built, and why

The core mechanism is a constitution: a human-written, natural-language document that lays out categories of permitted and restricted content with concrete positive and negative examples — explaining the properties of household chemicals is harmless, walking someone through purifying restricted chemicals is not. That constitution is used to prompt a helpful-only, unaligned LLM to synthetically generate large volumes of harmful and harmless example exchanges. This synthetic data is then heavily augmented — translated, paraphrased, varied through known jailbreak techniques — and expanded further via automated red-teaming, before being used to fine-tune two separate classifiers.

The input classifier is framed as a next-token prediction task over the user's prompt, flagging likely jailbreak attempts before generation even begins. The output classifier is the more novel piece: it streams a value-head prediction at every generated token, so it can halt generation mid-stream the moment the output starts trending harmful, rather than waiting for a complete response to judge. It's trained on a combined next-token-prediction and binary-cross-entropy-over-prefixes loss specifically so it can make a correct call on a partial, incomplete output.

That first version worked, but it was expensive and blunt in ways that only became visible under real production load. The second version, Constitutional Classifiers++, restructured the defense as a cheap-then-expensive cascade instead of running the full classifier on everything. A lightweight probe reads Claude's internal activations on all traffic, at near-zero cost, and only escalates the exchanges it flags as suspicious to the expensive full "exchange classifier" — which, unlike the V1 pair, looks at the input and output together as a single unit rather than judging each side separately.

The numbers

The V1 paper is unusually well-instrumented, and the comparison to V2 is the sharpest part of the story.

In automated red-teaming, V1 dropped the jailbreak success rate from 86% unguarded to 4.4% guarded — blocking roughly 95% of successful attacks. In production, that protection cost +23.7% inference compute overhead, a +0.38 percentage point absolute increase in refusals on harmless Claude.ai traffic, and roughly a 1.74% false-positive rate.

Anthropic also ran a human red-teaming exercise via HackerOne: 405 participants were invited from 800 applicants, of whom 183 counted as "active" (at least 15 queries and at least 3 triggered blocks). Bounties ran up to $15,000 per universal jailbreak found, with $95,000 total paid out, against an estimated combined red-teaming effort of roughly 4,720 hours (90% CI [3,242, 7,417]). Not a single participant successfully answered all 10 target CBRN-uplift questions — the best result was 6 of 10 — and most of the partial successes traced back to ambiguity in the grading rubric rather than an actual bypass of the classifier itself.

The V2 numbers, from one month of production deployment on Claude Sonnet 4.5 real traffic, are the headline result: refusal rate on harmless queries fell to 0.05%, an 87% drop relative to V1's refusal rate, while compute overhead fell to roughly 1%, down from V1's 23.7% — close to a 40× reduction in the compute cost of the safeguard. V2's own red-teaming logged 1,700+ cumulative hours across 198,000 attempts and surfaced only one high-risk vulnerability, a detection rate of 0.005 per thousand queries, with no universal jailbreak found against V2 as of the report.

The lesson

A safety or security classifier layered around a model — rather than baked entirely into its weights — can be measured, iterated, and cost-optimized independently, much like a WAF sits in front of a web application. The jump from V1 to V2 is the concrete proof: roughly 40× cheaper in compute, an 87% drop in false-positive refusals, and equal or better security, achieved without retraining Claude itself. That gap is also a general warning about first-generation guardrails — they tend to be wildly over-conservative on both compute and false positives simply because there was no production data yet to calibrate against. The fix that got Anthropic from V1 to V2, a cheap always-on screen that escalates only the suspicious cases to an expensive full check, is the same cascade/triage pattern used in fraud and abuse detection generally, and it's a strong, concrete answer to any system-design question about cost, latency, and accuracy trade-offs in safety-critical LLM infrastructure.

Sources