A new defense cuts attacks on AI agent tool inputs from 93% to 9%
The defense keeps 82.64% of usual task success, compared with 90.28% with no defense.
Researchers made AuthorityFlow, a defense for AI agents (software that uses tools for a user). It stops attacks that keep the task but change one input of a tool, such as the person who gets a message. With no defense, such attacks succeed in 93.06% of tests. With AuthorityFlow, they succeed in 9.03%. The authors report these results.
How it works
AuthorityFlow examines each tool input in 3 steps:
- It reads the user query and makes a set of fields that can set each input.
- It follows the value of each input back to its source.
- It accepts the value only when a field that the user query connects to that input set it.
The numbers
Attack success goes from 93.06% to 9.03%. Usual task success goes from 90.28% to 82.64%. The authors also made a new test set, FactAuthBench, for this type of attack.
What is not known
The paper gives no test on other attacks or on live systems. The defense does stop some usual work, as the lower task success shows.
Insight: Attackers move to tool inputs, away from tasks
Text in a repository made agents that write code propose a wide rule in 58% of tests. Changed names in code made agents write weak code in 22.5% to 56.0% of tests. This paper adds a new type: keep the task, change one input. It reads as a move around defenses that monitor only the task, and a defense must also find where each value came from. Tell your vendor to show that.
Sources
Posted
