In 72% of bad AI agent runs, proof of the error was ignored

Researchers read 95 runs with errors from six multi-agent systems in full. The runs are from two public test sets. They say that in 72% of the runs, the record held proof against the error before the task stopped. In 88% of those runs, no agent used it. The paper gives a method to find where an error starts and which control can catch it.

How it works

The team made an episode record for each fault. It links the fault to its start, how it moved, who saw it and the response. It also names the control that can catch it.

The numbers

• The median fault starts at 0.18 of the task.
• A system flagged a deviation in 31% of runs, but no response removed the bad condition.
• A step to put the agent back in a good condition was sufficient in 35 of 95 runs.
• 45% of Who&When bad runs ended in a crash at the model provider, then an answer that no agent made.

What is not known

The study covers six systems from public test sets. The paper does not say if the results hold for agents in live use.

Insight: Warnings exist but agents do not use them

The paper finds that proof of the error is in the record, but no agent uses it. A related test found that a tool result used again gave an average error rate of 0.649 across eight models. This reads as a missing part in the control software around the model. It must check and correct, not only record. Teams that make this layer can set the standard. Buyers must look for these steps.