Google Cloud shows six open AI tools for Kubernetes
Google tells that its llm-d 1.0 router made cache hits two times more, from 35% to 70%.
Google Cloud showed an open-source stack for AI at KubeCon + CloudNativeCon North America 2026. The stack has six parts for model training, model serving and AI agents, which are software that does tasks. It runs on Kubernetes, software that controls groups of computers. Teams can use it to operate open models on their computers.
How it works
OpenRL is a tool for training models with rewards. It uses the same processors for training and for rollouts, which are the tries of the model. llm-d 1.0 sends each query to the server that has the work from the turns before. gVisor keeps agent code that is not safe in a sandbox.
Three more parts add task scheduling, agent workflow control and PyTorch software for TPU processors.
The numbers
Google tells that llm-d made cache hit rates two times more, from 35% to 70%, in production. The value in the middle for the time to the first token fell by 70 to 85% on multi-turn agent work. More than 500 engineers from more than 25 organizations work on llm-d.
What is not known
Google gave no numbers for the other parts. It did not name the users in production behind the llm-d numbers.
Insight: Big vendors give away the agent control layer
AWS released the Strands Box sandbox in October, and Nvidia showed its OpenShell sandbox on 9 November. Nvidia also helped to found llm-d. This reads as a move to control the layer between models and processors. The default tool can control how other vendors work. Free tools can attach you to one vendor. Check how easily you can move.
Sources
Posted
