AI reviewers examine less when they know of a low error rate

A researcher wrote a paper on how to measure the time that AI removes from a task. The author says that a stopwatch is not sufficient. The time must be read together with the errors that stay in the work. The paper makes 1 decision number from these 2 values. It tests the number on public data and on a small test with AI reviewers.

How it works

The stopwatch view counts the time on the clock. The ledger view counts the time removed and the errors that stay. The author says that this view is correct. In the model, too much trust gives minutes on the clock but uses more minutes in total.

The numbers

• The author used the public data of the METR study of developers. The slowdown is 19% to 34%, and it changes with the method.
• In the GDPval workflow, the time to examine each output limits the speedup of GPT-5 to a maximum of 3.6x.
• AI reviewers that heard of a 10 times lower error rate examined less. An incorrect figure stayed in 22.5% of the briefs, against none.

What is not known

The test is small. It used AI reviewers and not persons. The same data show that AI helped on a small share of the issues, or on most of them.

Insight: A good AI record makes reviewers examine less

A participant at a closed roundtable in Singapore said that AI decreased the time of a report task from 6 hours to 5 minutes. Then the check by a person became a formality. This paper shows the same type of effect: AI reviewers that heard of a low error rate examined less. It reads as a good speed record that makes the check less strong, while the seller gets more and the errors show after.