A frozen guardrail check went from 1.01% to 15.12% incorrect alarms
After 16 public language model updates, the usual precision score moved by 0.0058 at most and hid the change.
Researchers froze a guardrail probe (a small check that flags dangerous prompts) and its threshold (the score limit that sets an alarm). They then measured it in 18 cells that include 16 public language model updates. With a 1% target, the threshold failed closed (it flagged too much) in 8 of 9 cells in the first phase. In a second phase it failed closed or open in 3 of 9 cells.
What the team found
The largest increase in incorrect alarms was from 1.01% to 15.12%. A benchmark rule for used checks flagged none of the cells. An estimate that uses no labels flagged 0.918 of the cells where the rate doubled or halved.
A fix and its limit
A new value for the threshold with stored benign prompts gives back the incorrect alarm rate. It needs no new labels. But if an attacker poisons 1% of these prompts, at most 0.017 of dangerous prompts are flagged.
What is not known
An attacker who controlled the update cut detection in one of three target categories in three of five seeds. The two checks missed it. A check with labels found it. The paper tests public models only.
Insight: An update can break a guardrail without a signal
Earlier papers show guardrails that stop from an attack (BRANCH, 9 October 2026) and from check errors (15 October 2026). This paper shows a third cause: a model update that nobody attacks. Safety checks do not stay set, and the update schedule of the model vendor decides when your guardrail stops. Leaders must plan the cost of a new test after each update.
Sources
Posted