A short plan carries most of the difference between 2 top agents
When the strongest agent got the plan of a weaker agent, it fell to the level of that agent.
Researchers made NextUnit-Bench, a test for coding agents that make models to predict when equipment will have a failure. It has 24 public datasets. Each agent has 2 hours to give a predictor that then operates on hidden units of equipment. AutoML, software that makes models, gets less than half the score of the top agent.
How it works
The agent makes a plan on the data of each machine. A plan in plain-text of some hundred words carries most of the difference between 2 frontier agents. Before the agent looks at the data, the plan it follows is of small importance.
The numbers
Of 30 published methods that the team made again on 4 datasets, only 6 beat the plan of the top agent. One rule can cut the reported error to half: a limit on the remaining life of test engines. A local open-weight model gave a result half as frequently with the top plans. Its score was no more than without them.
What is not known
It reads as if domain toolkits give no help. The abstract gives no scores for named agents. These are the results of the authors, in one field.
Insight: A good plan is not enough without a strong builder
In October 2026, llm-d 1.0 gave teams open-source software to operate AI models on their servers. In this test, a local open-weight model could not use the top plans: it gave a result half as frequently. The plan is cheap to copy, but this reads as control that stays with the strongest agents. For leaders, a plan from a strong agent does not make a weak model strong.
Sources
Posted