In a test of when to stop, 17 AI models got 59.5% at most
The authors report that more skill at tasks does not give an agent the skill to stop.
Researchers made AgentAbstain, a benchmark (a standard test) that checks if AI agents know when to stop. It has 263 paired tasks in 42 sandbox environments. Across 17 models, the top result was 59.5% of pairs correct on the task side and on the stop side. Gemini 3.1 Pro had this result.
How it works
Each pair has one task where the agent must work and one changed task where it must stop. The change is in the instruction, the tool or the environment. A tool, AbstainGen, makes the sandbox environments and the pairs with no help.
The numbers
The 17 models are in 4 agent harnesses (control software around the model). The authors report that the skill to stop is not the same as general skill at tasks. Three annotators find 94% to 98% of sampled tasks correct in design.
What is not known
The authors name one problem, post-hoc abstention. The agent does a step that it cannot make opposite, and only then sees that it must stop. The report gives no results for other models.
Insight: Two tests show that agents lack restraint
Other researchers made LITMUS. In that test, Claude Sonnet 4.6 did 40.64% of 819 high risk tasks. AgentAbstain shows the top model correct on 59.5% of pairs. Restraint is a different skill, and a good model score does not show that an agent is safe. Buyers must test restraint alone.
Sources
Posted