A new test shows that agent-written verifiers reject correct solutions

Researchers made VerifierBench, a new test of if AI agents can write a verifier. A verifier is code that accepts all correct solutions to a task and rejects incorrect solutions. The test has 222 tasks from six agent benchmarks and 4,993 hidden test cases. Across 19 models, normalized balanced accuracy is from 10.6 to 62.1.

How it works

An agent gets the instruction and the environment of a task and writes a verifier. The hidden test cases include other correct solutions and attacks. An audit with six checks, Verifier4Verifier (V4V), checks the source verifiers and test suites.

The numbers

Generated verifiers reject incorrect solutions more frequently than they accept correct solutions. This can make a verifier that is too hard look good. Most incorrect rejections come from verifiers that accept no solution to a task in the tests.

Why errors stay

Case studies show incorrect assumptions about solution formats, tool use and code changes. Agents test solutions that share the same incorrect assumptions, or do not make sure that a correct solution is accepted. The source names no top model.

Insight: A verifier is a good reward only if it accepts correct answers

In a different paper, a verifier that checks code against a correct specification increased proof verification from 66.1% to 93.5%. This paper shows that verifiers which agents write frequently accept no correct solution. The reward is only as good as its verifier, and a bad one can discard good data.