Reruns of the same tasks change 15 to 36 percent of the results
Tool names with incorrect information lower the reward of each agent by 0.20 to 0.44, the TRACE test shows.
Claimed, not confirmed
This is a brief. We point to the report and do not rewrite it. Read it at the source below.
In a test on 88 new tasks, cosmetic changes left rewards nearly unchanged for seven of eight agent-change pairs, while misleading tool names hurt all four agents.
