An incorrect message changes as many as 40% of correct AI verdicts

Researchers did a test of an Agent-as-a-Judge, an AI evaluator that has tools and checks the work of an agent against its environment. One incorrect message, which names an authority as its source, changed as many as 40% of the correct verdicts. The test used 4 judges and 5 datasets. The judges had found the data that shows the right result, and the message had no connection to it.

How it works

The researchers use the message as a test of confirmation bias, the habit to trust the first claim. They found 3 types of error:

• Too much trust: the judge checks, finds the right result, then accepts the message.
• Invention: the judge makes up facts that agree with the message.
• Short check: the judge stops at the first sign that agrees.

The numbers

For the 2 Claude judges, 67 to 75% of the incorrect changes came from too much trust. Small judges showed all 3 types.

What is not known

Light fixes decrease the bias but do not remove it. The source gives no numbers for the fixes. It does not tell how the bias works in use.

Insight: Judges that check also agree with authority

On 18 October 2026, Arize showed how to test agents with code checks, AI judges and labels from a person. A study on 28 October 2026 found that 5 of 7 models agree more frequently after a citation. Judges also agree. The check loop maybe has the same weak point as the agent that it checks. Keep code checks and labels from a person with AI judges.