The WebMirage attack hijacks web agents in 91.9% of tests
The attack can also get around 3 agent-level defenses.
Claimed, not confirmed
Researchers have made WebMirage, a framework that makes small changes to images on a webpage. The changes cause a web agent to select content that the attacker controls, and to do the related task in the browser. The researchers did tests with 4 agent configurations and 6 vision-language models on 2,250 tasks. They find an average attack success rate of 91.9%. The baseline with the most success had 17.4%.
Sources
Posted
