A weak AI agent got more from 29 guidelines than from all 199
The correct quantity of agent memory is a property of the model, not of the memory.
IBM researchers tell how they made memory for AI agents. It reads the work that the agent did before, writes short lessons and connects them into guidelines. It is open-source and is in the IBM CUGA agent and the watsonx Orchestrate platform. On the AppWorld test, gpt-oss-120b got 55.4% with about 29 selected guidelines and 51.2% with all 199.
How it works
The system gets 1 to 3 lessons from each operation of an agent. It connects them into a short set of guidelines. Each guideline records how many lessons it holds. The agent gets all guidelines or a selected part.
The numbers
- gpt-oss-120b started at 39.9%.
- DeepSeek-V3.2 did 9.5 points more of the tasks with all 191 guidelines.
- A strong model got more correct with more guidelines.
- A weak model got less from the full set than from a small set.
What went wrong
The authors give these problems:
- One guideline that many lessons made made the results less correct.
- Source records were removed when lessons were connected.
- An API error that no one saw made all tasks end with an error, but the record looked correct.
The 55.4% is the average of 2 operations.
Insight: More agent memory is not always better
Databricks tells that memory gave 3 times more correct enterprise tasks. The MemCut test found that agent memory keeps signs of secrets. This study adds that the correct quantity of memory is different for each model. More memory is the message in the market, but only a test on your model gives the answer. Builders must test the quantity of memory for each model.
Sources
Posted