Web agents send user data to scam sites in 54 to 93% of tests

Researchers made Scammer4U, a test with 91 scam sites and 10 safe matched sites. Web agents are AI software that read pages and fill in forms for a user. With no privacy rules, the agents sent the most important user data to the scam sites in 54 to 93% of tests. On the safe sites, they sent none.

How it works

The scam sites use social engineering. This is a method to make the agent send user data to an attacker. The test has 8 attack types and 16 site types. It uses 4 families of model.

The numbers

• Agents that told of a risk on the site sent very important data in 35.9% of sessions.
• Agents that told of no risk sent it in 66.1%.
• More prompt rules cut the leaks by very different quantities for each family.

The prompt rules did not stop the leaks.

What is not known

The agents had no defense in the test. The paper does not show how a complete system with other controls does. The authors want a check on each form that the agent sends. This check works apart from the logic of the agent. The paper gives no test of it.

Insight: Attacker pages reach agents at two points

A different test found attacker pages in 31% of AI search answers. This test shows that attacker pages also make agents send user data. One page is a low cost for the attacker, and the agent is a weak gate, so the defense must be apart from the agent. The next move is a check on each form that goes out. For you, the risk signal of an agent is not a control.