Microsoft shows how to backdoor an AI model with no training

Microsoft researchers show “Lazarus”, a method to hide a backdoor in an AI model. The method changes the weights, the numbers that keep what the model learned. The change affects fewer than 10 dormant neurons, which are parts of the model that rarely fire. A trigger such as a date change then makes an agent download and run attacker code.

How it works

The attacker makes dormant neurons detect a trigger. When one neuron fires, it forces a chosen word into the output. Some neurons together can make a full command. The attack has no training step. It must have write access to the model files and memory to operate the model.

The test results

The researchers say the model operates correctly and passes standard tests until the trigger appears. They say frontier agents can find such edits when you tell them to look. But no scan can prove that a model is safe. A reliable check compares the model with the initial, unchanged weights.

What is not known

Microsoft says its Foundry service ties each open model to its publisher. It says this protects customers who use models from trusted labs. The risk that remains is a trusted provider that is itself compromised. Weight scans, red-team tools and signed models are in development.

Insight: Backdoors have two forms and both pass tests

Earlier research put a backdoor in a model for less than $50, with poison in 20 percent of the training rows. Lazarus has no training step. Microsoft offers a defense: proof of the model source. This reads as a race to make the model source the control point, and the cloud that vouches for models gets the trust. Buyers must tell each supplier who can change the files.