A new benchmark finds that AI agents frequently do not release locks
Frontier models are not correct in more than 58% of closing steps.
CLASHarness is a benchmark of 600 tasks in which agents get reports that do not agree. In tests on 15 frontier models, agents could find inconsistencies, but success decreased when they had to repair the system. They frequently did not release shared locks or transfer tasks.
Sources
Posted
