Constitutional AI

How Constitutional AI trains a model

Bai and colleagues at Anthropic published the method in December 2022. The method has two phases. Each phase uses written principles.

Phase	What the model does	What trains the model
1. Supervised	Writes an answer, critiques it against a principle, writes a revision	Fine-tuning on the revised answers
2. Reinforcement learning	Writes two answers. A model picks the better one against a principle	A preference model learns from those AI selections and gives the reward

The second phase is RL from AI Feedback (RLAIF). The paper gives this critique instruction: “Identify specific ways in which the assistant’s last response is harmful”.

What Constitutional AI gives and what it costs

The 2022 paper and Anthropic’s May 2023 post give three results:

• Less human exposure to disturbing content. Anthropic says the method works “without needing lots of humans to view large amounts of disturbing, traumatic content”.
• The rules are on paper. “We can easily specify, inspect, and understand the principles the AI system is following”.
• Fewer evasive answers. The paper says the RLHF models it compared “are often evasive” on sensitive topics. Its Constitutional AI model was “similarly harmless but significantly less evasive”.

The method also has a cost. The authors say it can make it easier to deploy models “that have not been thoroughly tested and observed by humans”.

In practice

Anthropic uses constitutions for three purposes:

1. Claude’s constitution, May 2023. A list of principles from sources that include the UN Declaration of Human Rights, DeepMind’s Sparrow principles and Apple’s terms of service.
2. Claude’s new constitution, January 2026. Anthropic says the previous one was “a list of standalone principles”. The new one explains why. It gives four values in sequence: broadly safe, broadly ethical, compliant with Anthropic’s guidelines, “genuinely helpful”. Anthropic releases it with the CC0 1.0 license.
3. Constitutional Classifiers, February 2025. A constitution defines allowed and disallowed content. Claude makes synthetic data from it, and that data trains filters on the input and the output.

The numbers of the classifier test, from Anthropic (a vendor claim):

Measure, 10,000 synthetic jailbreak prompts, a version of Claude 3.5 Sonnet	Result
Jailbreak success with no classifiers	86%
Jailbreak success with classifiers	4.4%
Increase in refusals	0.38%
Increase in computer cost	23.7%

A public version also exists. In October 2023, Anthropic and the Collective Intelligence Project asked about 1,000 Americans to write a constitution on the Polis platform. They gave 1,127 statements and 38,252 votes. The result overlapped about 50% with Anthropic’s own list.

Trade-offs of Constitutional AI

The 2022 paper and Anthropic name these limits:

• The principles were ad hoc. The paper says its principles “were selected in an ad hoc manner for research purposes”.
• Too much training gives boilerplate. The paper found “Goodharting behavior”, for example “You are valid, valued, and cared for.” as the reply to a dangerous prompt. See reward hacking.
• Dual use. The authors say the ideas “have a dual use”, as most methods to control AI behaviour do.
• No guarantee. Anthropic says Claude’s outputs “might not always adhere to the constitution’s ideals.”

Where definitions disagree

Today the word “constitution” names different things:

• The 2022 method. Short written principles that guide AI feedback in training. The paper says it wrote 16 principles.
• The 2026 Claude document. A text that explains why and gives a sequence of values. Claude uses it to make synthetic training data and “rankings of possible responses”.
• Specifications of other labs. OpenAI’s deliberative alignment “directly teaches the model safety specifications”. It trains the model to “explicitly recall and accurately reason over the specifications before answering”. That is a different method, but the document has almost the same role.

A system prompt is not a constitution. The model reads a system prompt in each conversation. A constitution has its effect during training.

Questions and answers

What is the difference between Constitutional AI and RLHF?

Reinforcement learning from human feedback (RLHF) trains a model on human selections between two answers. In Constitutional AI, a model makes those selections for harmlessness, and written principles tell it which answer is better. This is RL from AI Feedback (RLAIF). In the 2022 paper, humans continued to label helpfulness. The method replaced only the human labels for harmlessness.

Is a constitution the same as a system prompt?

No. A constitution in Constitutional AI shapes the model during training. A system prompt is text that the model reads in each conversation. The two are written rules, but training on a constitution changes the weights of the model. A system prompt does not change the weights.

Does Constitutional AI stop jailbreaks?

Not on its own. Anthropic says Claude's outputs "might not always adhere to the constitution's ideals." Anthropic built different filters against jailbreaks, the Constitutional Classifiers. In a February 2025 post, Anthropic gave a test on 10,000 synthetic jailbreak prompts. The success rate fell from 86% to 4.4%. Anthropic says they "may not prevent every universal jailbreak."