The version label changes how much Claude Sonnet 5.5 checks an upgrade

Researchers tested if the version number of a software library changes how AI agents that write code work. They did almost 1,000 runs in 2 experiments, with GitHub Copilot CLI as the agent. The code was the same with each label. Some models checked less when the label said patch. In 4 runs, broken updates then got into the primary code.

How it works

The team made a small app that uses a made up library. The new version changed the name of a function, and the app broke after an upgrade. The agents got the same code with a patch label, a minor release label or a major release label. In a test of reviews, agents reviewed update branches and merged them if good.

The numbers

• Claude Haiku 5.5 merged a patch update with no check in 37% of runs, and a major release update in 7%.
• Claude Opus 5.5, GPT-6.1 Sol and GPT-6.1 Terra checked all updates.
• All 4 broken merges had a patch or minor release label.

Limits

The team made the labels incorrect on purpose. The apps were small, and the updates had no release notes. For Claude Sonnet 5.5 and GPT-6 Luna, the difference in the test of reviews was too small to be sure.

Insight: Agents trust small signals that anyone can set

Other tests showed that changed names in code made agents write dangerous code. In this test, a version label changes how much an agent checks. Each is a cheap signal that an agent trusts, and the person who sets the signal can steer the agent. For leaders: do not let a label select the steps that an agent skips.