About 1 in 17 agent skills tells the AI agent to skip approval

Researchers studied 1.53 million agent skills. A skill is a set of instructions that you can use again to extend an AI agent. About one in seventeen, or about 88,000 skills, has a line that lowers human oversight. Such a line tells the agent to continue with no approval. It can also tell the agent to skip checks. Of these lines, 70.1% apply with no condition.

How it was done

The team divided skills into units, such as sentences and code lines. They marked 2,097 units by hand. They then made a detector to mark the remaining units.

The numbers

• A small number of sources have most copies: 2.4% of skill families have half of all copies.
• Open-source scanners find a maximum of 46% of these lines.
• A large language model with the codebook of the team finds 61%.

What is not known

The abstract gives no count of damage that these lines caused. It also gives no data on how frequently an agent obeys such a line.

Insight: A skill is only advice, and one copied line can remove a check

The study finds that 2.4% of skill families have half of all copies of these lines. A Google kit has a skill that tells the agent to get consent, but no rule makes the agent obey it. This reads as a toll gate: who writes a skill that persons copy sets the rules for each agent that uses it.