Memory faults cut correct answers of AI agents by 40 points in a test

Researchers examined the effect when faults change which stored memories an AI agent can see. The agent can find the memory correctly. The stored content and the task stay the same. On the LongMemEval test, correct answers decreased by 40.54 percentage points between clean and corrupted runs. The work includes six providers and 2,080 fault pairs from 731 tasks.

How it works

The faults change which saved records go into the context, the text that the model reads. The records and the tasks do not change. The authors name this governance-state corruption.

The numbers

• In offline checks, corruption changed the records that the model saw in 96.25% of pairs.
• Oracle-backed enforcement made the records correct again in 100% of pairs.
• On the Memora test, correct answers decreased by 30.60 points.

What is not known

The authors report that effects of about 27.5 points across the full set are descriptive proxies, not checked effects. They report that the results apply only in the tests that they made. They want tests of memory control integrity with retrieval precision.

Insight: Agent memory is a weak point that needs its own check

In October, three studies looked at agent memory. One finds that memory poisoning is possible at five interfaces in a test of six LLMs. This study finds a 40 point decrease from corrupted memory control. The authors of AgentMemGate report that a gate increases correct tasks from 65% to 95%. Memory is a weak point in agents, and the repair is a check that is external to the model. Check memory control, not only retrieval.