Small changes in test data change the steps that agent guards stop
In one test, a marker change lowered the blocking rate by 8.33 points. The effect did not occur again in 2 other guards.
Claimed, not confirmed
Researchers tested the data that teams use to test guards for AI agents. They found that cues in the data can make labels easy to predict and can change guard decisions. On a set of 64 steps with Haiku calls, a change to the INFORMATION marker lowered blocking by 8.33 points. Opaque tool aliases increased it by 11.46 points. The effects did not occur again in Luna or in TS-Guard.
Sources
Posted
