Salesforce adds a model that gives an answer in 26 milliseconds

This month Salesforce launches Koa Decisions, a decision model and the next version of its routing model HyperClassifier. It reads a request and a list of alternatives that you give, then selects one in one step. Salesforce made it from Gemma, a Google model that has public weights, and it adds image and audio input.

How it works

The model reads the full request at the same time and gives its answer in one step. The answer is frequently one token, the basic unit of text, about one word. A large language model writes one token at a time.

Salesforce says a team can add an alternative with no retraining. It can be in production in hours, not months.

The numbers

Salesforce measured the routing of requests in Agentforce. The decision model took about 26 milliseconds. A general-purpose language model took about 1.4 seconds.

Salesforce gave no figures for correct results and no price.

What is not known

The next step from Salesforce is use of the model in builder tools and in an API for developers. It gave no dates. The price and the package can change, and availability can be different by region.

Salesforce gave no test against other decision models.

Insight: Salesforce joins a race to own the router

TypeSafe launched Jev in September. In October Microsoft gave Decision-1, and OpenAI gave its Decisions API, to customers. At this time Salesforce also has a model, and says customers do not select a model for each task. This reads as a race to control the router, because the router selects which model does the work. For buyers, the router is at this time a vendor decision.