Changed variable names made AI agents write code that is not safe
The agents wrote the weak point that the attacker wanted in 22.5% to 56.0% of tests. 7 analyzers found no weak point in the changed files.
Researchers show NameTrap, an attack on AI coding agents. The attacker only changes variable names in a Python file from a different source on the web. An agent that reads the file can then write code with a weak point that the attacker wants. The team made tests with 6 agents, 10 language models and 4 types of weak point. The agents wrote that weak point in 22.5% to 56.0% of tests, and in 75.0% for some agent and model pairs.
How it works
A coding agent on the side of the attacker changes the names in a file. It does this until the names agree with a name pattern that is connected to one weak point. It uses local tests to check that the file pushes a model to that weak point. The file has no instructions and no code with a weak point.
The numbers
The attack gave more weak code than random changes, neutral changes and the attack XOXO that came before. It also gave results with prompts in Traditional Chinese and with different models. A reminder about safety in the prompt did not stop the weak code each time.
What is not known
3 analyzers that use AI and 4 static analyzers, which read code but do not operate it, found none of the weak points that the attacker wanted. The paper shows no protection that always stops the attack.
Insight: Attacks on coding agents now hide in content and not in instructions
A previous paper put text in a repository. In 58% of tests, agents proposed the wide rule that the attacker wanted. NameTrap has no instructions at all, only new names. Filters that look for bad instructions find none here. Attackers seem to move to content that looks usual. Check the code that the agent writes.
Sources
Posted
