A study shows that accuracy is not a good test for AI agent teams

The authors of a paper show that accuracy is the incorrect test for a team of AI agents. The team uses a module that makes inferences about the other agents. In tests with 5 language models, accuracy showed that dangerous teams in Hanabi were safe. It also showed that good teams in task assignment were not safe. The authors give a different test, the Blind-Spot Gate.

The problem

The authors give 2 causes. The first cause is that the accuracy that a team must have is set by the cost of an incorrect step. The other cause is that the errors of a module are connected to each other.

Connected errors can help when their damage is in the same area. They can cause more damage when the damage adds up.

What the tests show

The tests were in task assignment, Level-Based Foraging and Hanabi. Accuracy was incorrect in the 2 conditions:

• It showed that dangerous deployments of language models in Hanabi were safe. In Hanabi, shared errors add up.
• It showed that good deployments in task assignment were not safe. There, only a small number of errors cost much.

The numbers

From GPT-5.4-mini to Claude Opus 5.5, the error on known teammates decreases from 32% to 11%. The risk that all 3 observers make an error on one teammate decreases much less than errors that are not connected can give.

The proposed gate

The Blind-Spot Gate includes the cost of errors, the pairs of errors and the measured dependence. It works without a live deployment. For a Bayesian module, accuracy gave an incorrect approval in 17 of 20 splits. The gate gave an incorrect approval in only 1.