A changed voice-to-text model made AI agents move $5,000 in a test

Researchers at Imperial College London changed voice-to-text models. The models write correct text for most persons, but not for one selected person. When that person speaks, the model writes text that the attacker selected before. In a sandbox test, AI agents read this text and moved $5,000. The changed Whisper-medium model did this in more than 9 of 10 clips of the target.

How it works

The attacker must have access to the model only while it is in training. This can be through a bad provider, an insider or a step with bad data. The model first receives training to tell voices apart. Then it receives training to connect one voice with the text of the attacker.

The numbers

For one target, Judi Dench, the backdoor fired in about 9 of 10 clips. It fired in less than 2 of 100 clips of other persons. For 4 other targets, it fired in 50% to 67% of clips. Agents moved the money in about 50% to 80% of tests, with different results for each model.

Not known

The study used 5 targets, 1 dataset of voices of known persons and 1 voice-cloning tool. Results changed from one training cycle to the next. The Qwen audio model kept much of its backdoor after clean training. Voice-disguise tools help only until the attacker trains with audio that has a changed voice.

Insight: Attacks move to the supply chain of models

In October 2026, other researchers put a backdoor in an open code model for less than $50. In the 2 studies, the attacker needs access only while the model is in training. The risk moves to the party that trains a model before you, because it can put in a trigger that usual use does not show. A model that you did not train must have a test.