ACaM reads tool code to find risks in AI agents
On 24 agents with manual labels, ACaM had a macro-F1 score of 98.3%, an accuracy measure.
Researchers show ACaM, a method that reads the code of the tools that an AI agent uses. It finds the functions of each tool in the code, and not the functions in its description. The team tested it on 43 agents that are in use. It found hundreds of structural weak points, and the team confirmed 10 with proof-of-concept exploits, which are test attacks.
How it works
Dangerous behavior can be in the code and not in the tool description. The authors say that this is capability laundering. ACaM finds facts in the code with static analysis. No AI output can change these facts. The AI adds only a description of the facts. ACaM then makes a graph of agent functions and of chains of tool use that an attacker can use.
The numbers
On 24 agents with manual labels, ACaM had a macro-F1 score of 98.3%. On injected threat chains it had a score of 0.980, which is 8 points above the baseline that is the nearest. The 43 agents included 9 multi-agent systems.
What is not known
The source gives no data on cost or speed. The team confirmed only 10 weak points with test attacks. It does not say how many of the other weak points are a risk in use.
Insight: A tool description is not proof of what the tool does
A related study put encrypted networks in model weights, and audits did not find them. ACaM shows a problem of the same type in tools: the description and the code are different. The two cases seem to show that the description of a system is not proof. The user must examine the code. Examine the code before you add a tool.
Sources
Posted