What matters in AI.

Subscribe

Arize shows how to test an AI agent with traces and judges

In its example, the reports that passed the test went from 5 of 16 to 48 of 48 after a change.

Laurie Voss of Arize shows that teams must record each step of an agent. Teams must score the records with code checks, AI judges and human labels. No other test result is given. For teams that make agents, this gives a number, not a guess.

Actionable reports Before the fix, 5 of 16 reports from the financial-analyst agent were actionable. After the fix, 48 of 48 were. Actionable reports BEFORE FIX AFTER FIX 5 of 16 48 of 48 Actionable Not actionable Judge: actionability eval
In Laurie Voss's workshop, an eval-checked fix took his agent's actionable reports from 5 of 16 to 48 of 48.

Sources

  1. How to Know If Your AI Agent Is Any Good: Evals With Phoenix & Claude — Laurie Voss, Arize AI|AI Engineerfinance.biggo.com
AI MATTER · NEWS · AI MATTER · NEWS ·18 OCT2026

Posted

Learn the terms in this story

More in Agents

All Agents news