Cloudflare releases Auto Router to select an AI model for each task

Cloudflare released Auto Router in beta, in its AI Gateway. A team sets the model to cloudflare/auto. A classifier then selects one model for each task. It rates the task for complexity, ambiguity, stakes and use of earlier context. Auto Router is free in the beta. Cloudflare gave no cost for the time after the beta.

How it works

The gateway makes a pool of models that can operate for the task. Then a classifier rates the task from 1 to 5 on four dimensions. A score matrix compares expected quality with cost. The reply names the model and the cause, and a team can audit the selection.

The numbers

Cloudflare made tests on 97 tasks, with three samples for each model. The result is ten points of reliability for about a third of the spend. In this benchmark, the router is below the baseline models.

What is not known

The classifier reads only the last turns, to a maximum of 8 KiB of text. Igor Pyrin says the turn that starts a task can look low in cost. Two baseline models in the table are not in the default pool. Without a session ID, one conversation can use different models.

Insight: The gateway gets control of the model selection

Cloudflare shows its decrease in the table. It also says the model pool can change. The router rates stakes from last turns only. This reads as a test: if teams let the gateway select the model, the gateway gets the control. Look for the alternative auto-best, which has no cost tradeoff.