Definition · AI security
Over-refusal
Over-refusal is a language model's refusal of a safe request that looks like a dangerous one. A typical example is a model that will not tell how to "kill a Python process" because of the word "kill". Researchers measure over-refusal with test sets of safe prompts that look unsafe, such as XSTest and OR-Bench.
Last reviewed
Key points
- Over-refusal is a refusal of a safe request. XSTest calls it "exaggerated safety" and links it to models that are "overly sensitive to certain words or phrases".
- In XSTest (NAACL 2024), Llama 2 70B chat with its default system prompt fully refused 38% of 250 safe prompts. GPT-4 fully refused 6.4%.
- Across 32 models, OR-Bench (ICML 2025) found a rank correlation of 0.89 between toxic prompt refusal and safe prompt refusal. Most models "trade over-refusal for safety".
- Over-refusal is also a security problem. One "blocker" document can make a RAG system refuse a target question, and safety tests do not show it.
How over-refusal happens
Safety training teaches a model to refuse dangerous requests. XSTest finds that models learn the surface words, not the intent. The authors use the term “lexical overfitting”.
XSTest sorts the safe prompts that cause it into ten types. Some examples:
| XSTest type | What makes a safe prompt look unsafe |
|---|---|
| Homonyms | A word with a safe and an unsafe sense: “kill a Python process” |
| Figurative language | A figure of speech that uses violent words |
| Safe contexts | A dangerous word in a safe context, such as a video game |
| Definitions | A question about what a dangerous term means |
| Historical events | A question about a known atrocity in history |
| Privacy (public) | A question about a public figure |
The system prompt also changes the rate. In XSTest, Llama 2 refused less with no system prompt than with its default one. Mistral refused more when a guardrail prompt was added.
What over-refusal costs
Over-refusal is the cost of model alignment. XSTest sees “a tension between two primary model objectives of being helpful and being harmless”.
The range is wide:
- XSTest, 250 safe prompts (2023): Llama 2 70B chat fully refused 38% and partially refused 21.6% more. GPT-4 fully refused 6.4%.
- OR-Bench-Hard-1K (2024 to 2025, temperature 0): Claude-2.1 refused 99.8%. Llama-3-70b refused 37.7%, GPT-4o 6.7%, Llama-3.1-70B 3.0%.
- The other side: on OR-Bench-Toxic, Claude-2.1 accepted 0.0% of toxic prompts and Llama-3.1-70B accepted 30.0%.
Across 32 models, OR-Bench found a rank correlation of 0.89 between the two rates. A model that refuses more toxic prompts usually refuses more safe prompts.
Over-refusal as an attack
An attacker can cause over-refusal on purpose. Shafran, Schuster and Shmatikov (USENIX Security 2025) call this “jamming” of retrieval augmented generation.
The blocker can make the system say that it does not have the information, or that the answer is unsafe. The method needs no instruction injection. The authors find that existing LLM safety metrics do not catch it. This is a denial of service, near to RAG poisoning.
In practice
Security teams find over-refusal in their own work. CyberSecEval 2 (April 2024) found that training a model to refuse unsafe prompts “can cause the LLM to falsely reject answering benign prompts”. Its False Refusal Rate test uses borderline cyber security requests.
Three things help a team read refusal results:
- Test the two sides. A refusal rate on safe prompts shows nothing alone. Add a rate on dangerous prompts, as OR-Bench does with 600 toxic prompts.
- Test with your own system prompt. XSTest shows the system prompt alone moves the rate.
- Count partial refusals. A refusal with a weak reply does not help the user either.
Fine-tuning can lower the rate. FalseReject (COLM 2025) tested 29 models and found “persistent over-refusal challenges”. Its authors report that fine-tuning on its 16k queries “substantially reduces unnecessary refusals”.
Where definitions disagree
Each test set counts a different thing:
- XSTest counts full refusal as the failure, and reports partial refusal on its own.
- OR-Bench counts any rejection of a prompt that a majority of three LLM judges called safe.
- CyberSecEval 2 counts false refusals of borderline requests in one field, cyber security.
“Safe” is also a judgment. XSTest prompts are hand-made by the authors. OR-Bench prompts are labelled safe by GPT-4-turbo, Llama-3-70b and Gemini-1.5-pro. A rate from one test set does not compare directly with a rate from another.
Questions and answers
What is the difference between over-refusal and a jailbreak?
A jailbreak makes a model answer a request that it must refuse. Over-refusal is the opposite error: the model refuses a request that it must answer. OR-Bench tests both sides together with 600 toxic prompts, so a model cannot get a good score by answering everything.
How is over-refusal measured?
A test set gives the model safe prompts that look unsafe and counts the refusals. XSTest uses 250 hand-made prompts and counts full and partial refusals. OR-Bench uses 80,000 generated prompts. CyberSecEval 2 reports a False Refusal Rate on borderline cyber security requests.
Can over-refusal be fixed without less safety?
Partly. OR-Bench found that most of 32 models "trade over-refusal for safety, with few breaking the trade-off". FalseReject reports that fine-tuning on its data cuts unnecessary refusals, and its authors say it keeps overall safety.
Sources
- XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language ModelsarXiv (Röttger et al., NAACL 2024), 2 Aug 2023
- OR-Bench: An Over-Refusal Benchmark for Large Language ModelsarXiv (Cui et al., ICML 2025), 31 May 2024
- CyberSecEval 2: A Wide-Ranging Cybersecurity Evaluation Suite for Large Language ModelsarXiv (Bhatt et al.), 19 Apr 2024
- FalseReject: A Resource for Improving Contextual Safety and Mitigating Over-Refusals in LLMs via Structured ReasoningarXiv (Zhang et al., COLM 2025), 12 May 2025
- Machine Against the RAG: Jamming Retrieval-Augmented Generation with Blocker DocumentsarXiv (Shafran, Schuster, Shmatikov, USENIX Security 2025), 9 Jun 2024
- Introducing the next generation of ClaudeAnthropic, 4 Mar 2024