Nvidia shows Agentic AI Stack to lower the cost of AI agents
In tests by Nvidia, the Switchyard tool kept an 80% task success rate at about a third of the cost.
At its AI Day in Seoul, South Korea, on the 9th, Nvidia showed the Agentic AI Stack. This is software for model selection, lower inference cost (the cost to operate a model) and security of AI agents, which are programs that do tasks. Jonathan Cohen, vice president of applied research at Nvidia, told that token use (small text units that models read and write) is two times more each month.
What is in the stack
The prompt of a user goes through control software to a correct model, and then to company data. The parts are:
- Switchyard selects a model by how hard the task is and by cost
- Dynamo, open source software, controls inference requests at data center scale
- OpenShell is a sandbox that stops agents when they do not follow company rules
The numbers
In tests by Nvidia, Switchyard kept an 80% task success rate at about a third of the cost. Nvidia tells that its Nemotron 3.5 Super VL model has a speed of 5x at most with almost the same or more results. The model was released last month. No external test was given.
What is not known
Some observers tell that small AI companies can have less room to be different, if platform operators control model selection. The source gives no cost for the stack.
Insight: Nvidia wants to be the layer between the models
In October 2026, Nvidia added Confidential Bare Metal for model makers, and Trend AI added the Nemotron model to its platform for customer data centers. At this time Nvidia adds model routing and security, which reads as a toll gate: one vendor at all layers can become the default. The cost to change vendor can be more than it looks.
Sources
Posted
