AI reviewers did not stop 1 in 6 bug attacks from AI attackers
The model GLM 5.3 made 13 of the 28 attacks that all 3 reviewers did not stop.
Researchers made CyberDual, a test where an AI attacker puts a bug in maintenance work on open-source projects. An AI reviewer must find and repair the bug. 3 reviewers found 85 to 91% of 1,088 attacks that operate, from 6 attackers. The attacker then put the bug in many commits from the source project. After the check, about 1 in 6 tries gave an exploit that operates.
How it works
The attacker puts a bug in a usual code change. Then the test operates the proof of concept on the code after the check. This is a small code that uses the bug.
The numbers
GLM 5.3 has open weights. Each person can get the model. It almost always accepts the attack task. It made 13 of the 28 attacks that no reviewer stopped.
Claude Sonnet 5 says no to 1 in 5 attack tries. But it makes attacks that are about as good as those of GPT-6-Sol. A model that says no to some tries keeps the skill to attack.
What is not known
When reviewers do not find an attack, Claude reviewers accept it most of the time. GPT-6-Sol most of the time repairs the incorrect code. A reviewer from a different vendor finds about 2 in 3 of the attacks that a Claude reviewer did not find.
Insight: AI checks of code and of papers are weak against attack
A previous test of AI tools that check papers found them weak when a person changes the input to attack them. CyberDual shows the same for code checks. An AI check seems to be one weak point that attackers can point at. Use checkers from different vendors. A reviewer from a different vendor found about 2 in 3 of the attacks that a Claude reviewer did not find.
Sources
Posted
