A paper shows a reply time of 150 milliseconds for Jev
For a million input tokens (units of text), the price of Jev is $0.042 and the price of GPT-6 Astra is $10.
On 15 September, the startup TypeSafe AI released Jev, a decision model. A decision model reads a text and selects one answer from a set of alternatives, and this is fast. The company tells that Jev has a low price and cannot hallucinate (give incorrect facts). Authors at Carnegie Mellon University found a median reply time of 150 milliseconds. GPT-6 Astra of OpenAI had a median reply time of 1.89 seconds.
How it works
A large language model (LLM, a model that writes text) writes one token at a time. A decision model makes one step and gives one answer. It must select from alternatives such as A, B or C, and it gives a value for each alternative.
The numbers
- GPT-6 Astra has a price of $50 for a million output tokens. For Jev it is free.
- Michelle Chen of Cloudflare tells that 80 percent of typical LLM tasks can go to such a model.
4 large firms, such as Amazon and Microsoft, released equivalent models in weeks.
What is not known
TypeSafe AI did not tell how Jev gets its speed. No test result is given for how frequently Jev gives the correct answer. Chen tells that decision models are almost a new name for classifiers. Classifiers sort data into groups, as spam filters do.
Insight: Rivals copy a fast model in weeks, so speed is not a lasting edge
Workday uses Jev to select the screen parts of its Sana assistant. Microsoft tells that its Decision-1 model is 2.8 times as fast as Jev in its test. Jev set the category, and the edge looks set to move to a model that is fast and has a low price. Test 2 on your task before a long contract.
Sources
- AI Companies Develop Decision Models to Avoid Hallucinationsspectrum.ieee.org
Posted