BenchDoor puts a backdoor in models through benchmark scores

Researchers showed BenchDoor, a type of data poisoning attack. Teams make training data to make benchmark scores go up. BenchDoor uses this: its data makes the score go up and also puts in a backdoor, a step that users cannot see. On Qwen and Gemma models, the held-out attack success rate was 82.33% to 100.00% at poison rates of 2% to 10%.

How it works

The attacker has 2 methods. The attacker changes the training data or test data of a benchmark. Or the attacker gives a different training dataset and does not change the benchmark. The researchers give 5 attack scenarios. These include the use of changed test examples as training data. They also include similarity-based removal of data that is almost the same as test data.

The numbers

The tests used multiple-choice Q&A, text sorting and open-ended Q&A. The benchmark score went up in all 30 configurations. After synthetic data generation and that removal, the attack success rate was 90.33% or more.

What is not known

The paper gives no example of this attack in use. The tests used only Qwen and Gemma models. No other group did the tests again.

Insight: A helpful dataset can be the way in for a backdoor

Researchers made a backdoor in an open Qwen model, with poison in 20% of data rows, for less than $50. BenchDoor uses only 2% to 10% poison and comes with a score that is more. This reads as a move of the attack from the model to the dataset, because teams use data that makes their score go up. Teams that select data only by score have this risk.