MARS cuts the wait of AI agents on GPUs with small memory

The authors of a new paper show MARS, a system that operates self-hosted AI agents. It operates on a GPU, a processor that does the heavy work of an AI model. Agent work gets long. The key-value (KV) cache holds the past work of the model, and it can fill the GPU memory. MARS stops this.

How it works

MARS has 2 parts. An admission controller selects the workflows to start. It uses the start of the work and the free cache space. A scheduler in the engine completes the open turns first.

The numbers

The authors give these results from their tests:

• Mean latency is 1.59 to 7.56 times lower than the fastest baseline.
• With OpenCode, Codex and OpenHands, mean workflow time is 1.17 to 1.87 times lower.
• The source code is open.

What is not known

The paper names no models and no GPUs. No other group gives test results. The paper gives no data on money spent.

Insight: Memory now limits how many agents operate

In October 2026, vLLM added a two-way cache move that cut the wait by a maximum of 3.3 times. In November 2026, AWS gave each agent session 32GB of memory. MARS controls which agents get memory at all. Memory now limits agents, and the scheduler seems to set how many agents one GPU serves. Find where your agent memory goes before you get more GPUs.