Language model guardrails

How a guard model checks text

A guard model is a second model adjacent to the primary model. It does not answer the user. It reads text and gives a verdict.

The steps in the Llama Guard paper (Meta, December 2023):

1. Put the policy, a list of risk categories, in the guard model prompt.
2. Add the text to check: the prompt, or the prompt and the response.
3. The guard model writes “safe” or “unsafe”, and the categories that the text breaks.
4. Use the probability of the first token as a score.

NVIDIA NeMo Guardrails uses the same idea with a general model. Its self check rail tells the model to answer “yes” to stop the text.

What a guard model gives security teams, and its weak points

A guard model finds harmful text that a word list cannot find. It is the AI-based part of AI guardrails, and it can check input and output.

But a guard model is a language model, with the same weak points:

• Evasion. Meta says Llama Guard 3 “may be susceptible to adversarial attacks or prompt injection attacks”.
• Policy text. Google says ShieldGemma is “highly sensitive” to the description of safety principles that the user gives.
• Out of date facts. Meta says some categories, such as Elections, need current facts.

In practice

Four known guard models:

Model	Publisher	Base and size	What it labels
Llama Guard 3	Meta	Llama 3.1, 8B	Prompts and responses, 14 categories (13 MLCommons hazards plus Code Interpreter Abuse), 8 languages
ShieldGemma	Google	Gemma 2, 2B, 9B or 27B	Prompts and responses, 4 harm types
WildGuard	Han et al. (2024)	Open model	Prompt harm, response harm and refusal, 13 risk categories
Prompt Guard	Meta	mDeBERTa, 86M backbone	Input only: benign, injection or jailbreak

Prompt Guard is a small classifier, not a generative model. Meta recommends that you fine-tune it on your own application data.

What the research measured

Attack success rate (ASR) is the share of attacks that got through. Five results from 2024 to 2026:

• One good result. WildGuard cut jailbreak success in one test interface “from 79.8% to 2.4%” (Han et al., June 2024).
• Small change on strong attacks. Wang and colleagues tested nine jailbreak attacks on Llama 3 8B Instruct (June 2025). Average ASR was 0.238 with no guardrail and 0.227 with the first Llama Guard on input. Against X-Teaming, ASR stayed at 0.910. GuardReasoner on input gave the lowest ASR, 0.135.
• Character tricks. Hackett and colleagues tested six guardrails, with Azure Prompt Shield and Prompt Guard (April 2025). Emoji smuggling got 100% ASR on 476 prompt injection and 78 jailbreak prompts.
• Word swaps. In the same study, the TextFooler attack averaged 46.27% ASR on prompt injections and 48.46% on jailbreaks.
• Attacks on the guard model itself. Zhou and colleagues (June 2026) made reasoning guard models write 13 to 63 times more tokens. Latency in agent systems went up by a maximum of 148 times. One poisoned document can fill a shared guardrail service.

Trade-offs: three types of model check

Type	Strength	Limit
Guard model made for safety (Llama Guard, ShieldGemma)	Safety labels, policy in the prompt	More GPU memory, incorrect results on new data
Self check (NeMo Guardrails)	Uses the LLM with a custom prompt	Results change with how correctly the model obeys instructions
Small classifier (Prompt Guard)	Very small model (86M backbone), one label set	Needs fine-tuning, “not immune to adaptive attacks”

Wang and colleagues found that “no single guardrail excels across all dimensions”. LLM-based guardrails also “impose a greater memory burden”. NVIDIA says a fuller check prompt “uses more tokens and adds more latency”.

Where definitions disagree

The words “LLM guardrail” have two meanings in the research.

• Any guardrail around a language model. Hackett and colleagues use “LLM guardrails” for six systems. One, NeMo Guard Jailbreak Detect, is a random forest classifier, not a language model.
• A guardrail that is a language model. Meta calls Llama Guard “an LLM-based input-output safeguard model”. This page uses the second sense.

The numbers can also mislead. In August 2026, Zhang and colleagues measured how many refusals a guardrail itself caused. The share was 0%, 41 to 45%, or 99% in three tests that a standard report shows the same.

A request sent as an image went through a text guard with no check. Before you trust a guard model score, find what the guard model saw.

Questions and answers

What is the difference between a guard model and a content filter?

A content filter uses fixed rules, such as a word list. A guard model is a language model that reads the text and compares it with a policy in its prompt. You can change the Llama Guard categories with no new training. A guard model finds more meanings than a rule. It also costs more to operate, and attackers can trick it.

Can a language model guardrail stop prompt injection?

Only in part. Meta says Prompt Guard is "not immune to adaptive attacks". In April 2025, Hackett and colleagues got 100% evasion of six guardrail classifiers with emoji smuggling. A June 2025 study by Wang and colleagues found guardrails catch injections "modest at best". Use a guard model as one layer, and also limit what the primary model can do.