What matters in AI.

Subscribe

Agents that change skills with no feedback get 15 points less

After 30 rounds of changes, all models had lower results on all benchmarks, by 15 points on average on SkillsBench.

Researchers examined agent skills, which are documents where experts write down how to do a task. Models more frequently change these skills. The team let models change expert skills for 30 rounds with no task and no feedback. Agents with the new skills got 15 points less on average on SkillsBench. The authors also give SkillDiff, which adds 4.8 points.

How it works

Each change moves a skill to the text that the model writes from the subject. The knowledge that is not in this text is the first to go. SkillDiff marks this knowledge in the skill. It does not change the skill text.

The numbers

  • All models in the test had lower results on all benchmarks.
  • The average drop was 15 points after 30 rounds.
  • When the team put the knowledge back, much of the task score came back.
  • SkillDiff increased the task score by 4.8 points on average, with no feedback.

What is not known

The text does not name the models. It does not tell if a different group did the test again. The authors report all results.

Insight: Agents that change their skills lose in two ways

A study found that agents that got new skills from user feedback had an attack success rate that was 14.0 to 48.8 points more. This paper shows that change with no feedback makes skills less good. A model that changes expert text has a cost with feedback and with none, thus the expert version is the part to keep. Test each new version against it.

Sources

  1. How Feedback-Free Self-Evolution Reshapes Agent Skills: A Systematic Studyarxiv.org
AI MATTER · NEWS · AI MATTER · NEWS ·20 NOV2026

Posted

Tags

Learn the terms in this story

More in Research

All Research news