Test shows that defenses miss attacks on the decisions of AI agents

Researchers made a benchmark for indirect prompt injection. This is text in content that an AI agent reads, which tries to control the agent. The 90 tests use 3 tool environments and 5 language models. The attacks do not replace the task. They change a decision that the task must have, for example an incorrect refund policy in the proof of a buyer.

How it works

The test uses authority to say what an attack is. Content is an attack when it tries to change a decision that its source cannot make. Each test is for a maximum of 6 parts of the agent decision, such as its rules or its data about the condition.

The numbers

A defense that finds and removes text decreased attacks on the objective to 2.4% success. It left attacks on observations at 9.5% and on the condition at 12.1%. Without defense, attacks on rules, steps and the condition had a higher average success than attacks on the objective.

What is not known

The test uses 5 models and 90 items. It does not name a defense that works against all targets. The authors tell that tests which only move the task to a different task can show defenses as better than they are.

Insight: Defense scores depend on which attacks the test uses

In October 2026, a public challenge found that Meta SecAlign, which has training against prompt injection, had a 32% attack success rate. Other researchers made attacks work at a rate of 96%, where tests before found almost none. Each new test shows a defense that is less strong than the test before showed, and the company that selects the test selects the score. Tell vendors to give results on attacks that change values.