AOS-Bench shows coding agents lose up to 23 points on stacked work
Two agent systems 0.6 points apart on single tasks delivered 54% and 68% of the same tasks when the work came as a stream.
Researchers made AOS-Bench, a test for coding agents (AI that writes and runs code) that act as long term assistants and get a stream of work. A data job and a bug fix can wait on the same machine. Older tests score each task alone, but AOS-Bench lets checked tasks arrive over time with shared CPU and memory limits. On heavy workloads, each of five systems loses 13 to 23 points.
How it works
The test treats the agent system as a workload manager. It runs the same tasks one at a time to make a reference score for each system. A system is a model with the software around it.
The numbers
Serial dispatch leaves 73% of heavy tasks unclaimed. A fixed limit of two tasks at one time is within four points of the best agent managed policy. Median CPU use stays below 30% while tasks wait.
What is not known
The source does not name the five systems. It does not say if the results hold on other machines. The authors say that workload management is its own skill for agents.
Insight: Agent scores break when the setting changes
On 15 October 2026, EvoCowork showed agent increases of 4.90 points on one file type but only 0.03 on mixed tasks. AOS-Bench shows a like break for streams of work. Single task scores seem to overrate agents, and a simple fixed limit can beat agent self management. Builders can test in their own load.
Sources
Posted
