Failure labels for AI reasoning change with the sequence of the tests
In all six test sequences, the share of one failure type was between 26.4% and 76.4%.
Researchers studied how papers label the reasoning errors of large language models (LLMs), and they measured one failure type, Compositional Collapse. They used a public math benchmark (a standard test) with 2,316 graded records, twelve models and eleven languages. The reported share of this failure depends on the sequence of the classifier tests, and it was between 26.4% and 76.4%.
How it works
In this failure, a model sees a necessary part of a problem, then drops it or uses it incorrectly. In 50.1% of failures, the signs fit more than one category. This shows that the sequence of the tests sets the result.
The numbers
Native speakers agreed with the AI inspector at kappa 0.932 on correct answers. Kappa is a score of agreement where 1.0 is full agreement. On this failure type they agreed at only 0.093.
What is not known
The same inspector model, given the human instructions, rose from 0.147 to 0.636 against one human. The prompt caused most of the difference. The model is not the cause.
Insight: The person who writes the grader prompt sets the score
An earlier tool, ResolveLab, grades agents from server records, not from their claims. This paper finds that an AI grader changes its labels when the prompt changes. The grader is a player, and its instructions seem to set the result. A test that uses a shared human sample gets more trust than a test that uses one AI.
Sources
Posted
