What matters in AI.

Subscribe

A frozen guardrail check went from 1.01% to 15.12% incorrect alarms

After 16 public language model updates, the usual precision score moved by 0.0058 at most and hid the change.

Researchers froze a guardrail probe (a small check that flags dangerous prompts) and its threshold (the score limit that sets an alarm). They then measured it in 18 cells that include 16 public language model updates. With a 1% target, the threshold failed closed (it flagged too much) in 8 of 9 cells in the first phase. In a second phase it failed closed or open in 3 of 9 cells.

What the team found

The largest increase in incorrect alarms was from 1.01% to 15.12%. A benchmark rule for used checks flagged none of the cells. An estimate that uses no labels flagged 0.918 of the cells where the rate doubled or halved.

A fix and its limit

A new value for the threshold with stored benign prompts gives back the incorrect alarm rate. It needs no new labels. But if an attacker poisons 1% of these prompts, at most 0.017 of dangerous prompts are flagged.

What is not known

An attacker who controlled the update cut detection in one of three target categories in three of five seeds. The two checks missed it. A check with labels found it. The paper tests public models only.

Insight: An update can break a guardrail without a signal

Earlier papers show guardrails that stop from an attack (BRANCH, 9 October 2026) and from check errors (15 October 2026). This paper shows a third cause: a model update that nobody attacks. Safety checks do not stay set, and the update schedule of the model vendor decides when your guardrail stops. Leaders must plan the cost of a new test after each update.

Incorrect alarms rose from 1.01% to 15.12% Two bars on one scale: before 1.01%, after 15.12%, about 15 times longer. The usual precision score moved 0.0058 at most, a barely visible bar. Based on 16 public model updates, 18 cells. INCORRECT ALARMS, LARGEST INCREASE 15.12% BEFORE 1.01% 1% TARGET AFTER 15.12% Usual precision score moved 0.0058 at most 16 public model updates, 18 cells
In one test, a frozen guardrail check's incorrect alarms rose from 1.01% to 15.12% after a model update, while the usual score barely moved.

Sources

  1. Operating-Point Failure of Frozen Guardrail Probes Under Backbone Updatesarxiv.org
AI MATTER · NEWS · AI MATTER · NEWS ·17 NOV2026

Posted

Tags

Learn the terms in this story

Guides for this story

More in Security

All Security news