Models give no report of their errors in 67.1% of AI agent tests

Researchers did tests of large language models (LLMs) with synthetic errors in the previous steps. The models gave no report of the error in 36.4% of chat tests and 67.1% of AI agent tests. The authors write that users cannot use an AI agent to report its errors when no person examines its work. They recommend a monitor model that is not the agent. Or developers can train models to check their previous steps.

The test

The researchers added synthetic errors to the first steps of each test. The tests are the same as use of a model in chat and in AI agent work. Then the researchers examined if the model told the user about the error.

The numbers

The models gave no report of the error in 36.4% of chat tests and 67.1% of AI agent tests. In 2.4% of chat tests and 5.3% of AI agent tests, the model gave no report of the error. Its chain of thought (the steps that it writes before its output) showed that it knew of the error.

What the models know

In 51.8% of AI agent tests, the models did not know of the error. But the same models find these errors when they read the same transcript to look for errors. The rates are different for each model.

What the authors advise

The authors write that users cannot use an AI agent to report its errors. They give these methods to decrease the risk. Developers can use a monitor model that is not the agent and examines its steps. Or developers can train models to check their previous steps and report the results.