What matters in AI.

Subscribe

Research

Agent memory systems remove a fact correctly in at most 1.5% of tests

In the Bemula tests, a system left the data open to extraction by basic attacks or removed much more than the user wanted.

No memory system passed more than 1.5% of forget requests A grid of 100 squares with 1.5 filled red shows the best pass rate for forget requests end to end. Most systems passed none. The test used 6 memory systems, 3 backbone LLMs and 135 forget requests. FORGET REQUESTS PASSED END TO END at most 1.5% Most passed none. 6 MEMORY SYSTEMS 3 BACKBONE LLMS 135 FORGET REQUESTS
In the Bemula benchmark, no AI agent memory system passed more than 1.5% of user requests to forget something, end to end.

Sources

  1. Can Agents Forget When Asked? A Benchmark for Memory Unlearning in LLM Agentsarxiv.org
AI MATTER · NEWS · AI MATTER · NEWS ·14 OCT2026

Posted

Tags

More in Research

All Research news