A prompt injection detector does not work in agent tests
On the AgentDojo benchmark, Attention Tracker found attacks at a chance level for most model families.
Claimed, not confirmed
A team of researchers tested Attention Tracker, a tool from 2025 by Hung and others that finds prompt injection attacks. The tool measures the attention of a model between the user task and the data. Its makers claimed that one threshold works on all datasets, but the new tests on AgentDojo show that this is not correct.
How it works
Attention Tracker must have access to the inner parts of the model. The team made AttnDA. It replaces the one threshold with a classifier, a rule from training data, and uses the score of each attention head.
The results
On AgentDojo, long tool outputs break the basic assumptions of Attention Tracker. It gave a near chance level result for most model families. AttnDA found more attacks on 4 benchmarks, but the team gave no score.
What is not known
Results on AgentDojo stay weak for the 2 methods. The classifier must have training data from the same use. No other group tested the results.
Insight: A detector that works in one test can fail in agent work
In a different study, a prompt-level defense cut the mean attack success rate from 58.2% to 14.3% on 19 models. In a public challenge, the rate was 47% for GPT-5 mini and 58% for GPT-4o. This detector reads as a sign that results on simple tests do not move to agents in use, and the buyer must get the proof. Test on your own data.
Sources
Posted
