Prompt injection guards give incorrect alarms on 92% of some contracts
A cascade of two language models is 6.5 times cheaper than one frontier model and comes near its results.
Researchers tested eleven open weights guards on two sets of legal contracts. A guard is a tool that checks a document for hidden instructions that try to control an AI system. No one had published an incorrect alarm rate for a guard on contracts. At document level, the rate was as high as 92.39%. The authors say that existing guards struggle on this task.
How it works
Guards are in front of systems that read documents. The authors tested the eleven guards on full contracts. They then did controlled probes to find where each guard fails.
The numbers
Two guards did worse than a random guess. No guard beat a baseline of five simple text rules. Each guard had a score of zero on a second threat, claims of scope without approval. Prompted language models fixed the tested problems.
What is not known
The paper tests contracts only. It does not say how guards do on different types of document.
Insight: Guards fail in two ways, so test them on your documents
Earlier tests found attacks that work in 95.41% of the tests. Meta SecAlign, a defence with training, had a 32% attack success rate. This paper shows the other error: guards flag clean contracts. A guard seems to trade missed attacks for blocked work, while general models do better. Tell vendors to give the incorrect alarm rate on your own documents.
Sources
Posted
