One short suffix flips 43% of unseen documents for the Jev model
One suffix from one task flips 90 to 100% of samples in other tasks, and the attack finds it in minutes.
Researchers made FlipJev, a fast attack on Jev, a typed decision model. This type of model gives a decision in one fast query, and not free text. On 12 datasets, one short suffix for each dataset flipped 43% of unseen documents. The attack uses only query access to the model. It uses no attacker language model.
How it works
The attack adds a short text, a suffix, to the end of an input. It uses only the answers of the model to find the text. It finds a suffix that flips answers in minutes.
The numbers
- One suffix for each dataset flipped 43% of unseen documents.
- A suffix from one task flipped 90 to 100% of samples in other tasks.
- Usual prompt injections changed no correct answer in 10,240 paired tests.
What is not known
The authors tested Jev only. The abstract does not tell how other typed decision models do.
Insight: The speed of a decision model is also the speed of the attack
Microsoft released Decision-1, a model that is 2.8 times as fast as Jev in its latency test. A different test found that attacks change 46.3% of correct answers by Jev. Models of this type are supplied for speed, and the same speed lets an attacker try many inputs fast, so speed maybe is both the main benefit and the weak point.
Sources
Posted
