An attack makes text in an image clear only after the image shrinks

Researchers show an attack on vision-language models, which are AI models that read images and text. The attack puts a text command in an image, and the model can read it only when the system makes the image small. After 3 steps of its loop, the attack has a 58.75% success rate on 4 closed models, and other methods have 41.08% or less.

How it works

The loop changes the image in many steps. At 512 by 512 pixels the model cannot read the command, but when the image is small, it can. The team does not use the inner parts of the model.

The numbers

• When the image is not as small, the success rate decreases to 5.60%.
• In a test with web agents, the attack makes about 4% of agents follow the command.
• The detector ScaleGuard decreases the success rate from 61.4% to 24.3%, and it rejects no clean images.

What is not known

The images make the agent see more of the command, but only some agents follow it. Usual changes to an image give small protection. The paper tests only closed models and one set of test items.

Insight: Attackers now tune attacks to each system

This attack uses a loop of changes and tests on the image, with no access to the inner parts of the model. A related test set has a maximum of 95.41% for attacks that change with browser-agent feedback. Because attacks change to fit each system, a defense must have tests against attacks that change.