Live tool data cuts agent lost time from 4,252 ms to 131 ms
The agent completed its task on the first try in 95.8% of tests, against 69.4% with no data.
Researchers made Agent Harness Awareness, a method where the host software gives an AI agent a small, new report on tool condition before a primary step. The host keeps the control to measure and to send the last instruction. In 216 tests with 3 models, the report cut the mean lost time by 96.9%, from 4,252 ms to 131 ms.
How it works
Tool descriptions tell what a step can do. They do not tell if the tool is in good condition at this time. The host software knows this from monitors and logs, but the model does not. The authors name this difference operational blindness.
The numbers
The models were GPT-5.6 Sol, Claude Sonnet 5 and Gemini 3.7 Flash. The share of tasks that the agent completed on the first try went from 69.4% to 95.8%. In a different test of 27 live decisions with faults, the task was complete in 9 of 9 tests, against 5 of 9 with no report.
What is not known
The tests include only tools and network. The authors leave runtime condition, hardware load, cost and retries for later work. The sample is small, and the authors made the benchmark.
Insight: The software around the model decides how well an agent works
In a report from before, one change to the harness increased a score by 11.5 points with no change to the model. In a different report, a harness made models less safe. The control software, not the model, sets both the result and the risk, and the team that makes it gets the control. It reads as a signal that the selection of a harness is more important than the selection of a model.
Sources
Posted
