A new training method cuts prompt injection success to 8.8%

Researchers published SecPO, a method to train a language model to resist prompt injection. In this attack, an email, a document or a web page gives hidden commands to an AI agent. The best earlier defended model, Meta-SecAlign, had attack success rates near 100% against adaptive attacks. The new model had 8.8% in one agent test.

How it works

The team made two changes. They train the model on strong attacks that they make with a search of 240 tries. They also use a new loss function, SecPO, in place of DPO. The authors say DPO keeps the model near a start model that is not safe.

The numbers

The attack training cut attack success 1.5 to 14.5 times against Meta-SecAlign, across three attacks. SecPO cut it 2.4 to 4.6 times more than DPO. The authors say the model had no noticeable loss in nine benchmarks.

What is not known

All results are from the authors. The source names no test by an outside group. It does not say how the model does against new attacks.

Insight: A low attack number holds only for the attacks tested

Our earlier stories show Meta SecAlign at 32% in a public challenge. They also show new attack methods at 95% to 96% against defended models. This paper lowers the number again. This reads as a race where each defense is tested against the attacks known at that time, and the attackers move next. A low number on a vendor page is a start, not proof.