guardscope tests which tasks an AI agent control checks

Researchers released guardscope, a tool that measures which tasks of an AI agent a runtime control checks. A runtime control is software that checks agent tasks while they occur. The authors say that a control sees only the tasks at its own point. An agent has more paths than tool calls. The summary gives no coverage result.

How it works

guardscope has two steps. First, it reads the source of a control and predicts which of 15 task paths the control can check. Second, it runs a fixed workload and puts each task in a group: checked, bypassed or prevented. It uses kernel records (the core of the operating system) as the ground truth.

The proof

The team did tests with two harnesses (the software that runs an agent) and controls in use. These include the agent governance toolkit of Microsoft and agentgateway. The summary gives no result for the controls.

What is not known

It is not known how much each control misses. It is not known if control makers will publish their coverage. The authors say that a maker can publish at this time which paths its product checks.

Insight: Speed claims are not coverage claims

Solcis says that its Policy Engine gives a decision on each agent task in less than 200 milliseconds, and that it works with the Microsoft toolkit. That is a speed claim. guardscope tests the same toolkit for coverage. This reads as a change in what a buyer must have: not how fast a control decides, but which paths it does not see. A vendor with no kernel proof can get fewer deals.