Study names 9 types of prompt injection attack

Researchers made a set of 9 dimensions that show how prompt injection attacks are different. Prompt injection is text in data that makes an AI model follow an instruction that the user did not give. They used the set on 6 live vulnerabilities and 9 test sets. Most test sets include only a small part of the space of attacks.

How it works

The 9 dimensions give the instruction in the attack, its relation to the task of the user, and how it shows in the target system. The same instruction can be an attack in one context and a query of the user in a different context. Each dimension changes which defenses can find or stop an attack.

The numbers

All together, 13% of pairs of categories are not tested. Most are in categories that have a small number of attacks. The set of dimensions also marks test sets with no manual work and makes new tests for the pairs that have no test.

What is not known

The abstract gives no defense scores on the new tests. It does not give how much the new tests change the results of known defenses.

Insight: An attack score has no use without the types it includes

One challenge gave 208,095 attack prompts, and GPT-4o had a 58% attack success rate. A different test set has a maximum success rate of 95.41% for attacks that use feedback from a browser agent. Each number includes only the attacks in its test set, and it reads as a score for a small area. Find out which types a test includes.