Positron says two-thirds of AI workloads are inference
Positron says inference was 15% of AI work in 2023 and is about two-thirds in 2026.
Cameron McCaskill, head of sales at Positron AI, spoke at SC26. McCaskill says about two-thirds of AI workloads are inference, which is the use of a model to give results. The share was 15% in 2023. The figure is the view of Positron, which makes inference servers. In September it raised $875 million at a value of $5 billion.
How it works
Positron makes inference servers with standard memory, not high-bandwidth memory. McCaskill says high-bandwidth memory is in short supply and has a high cost. One server uses about 4,500 watts, and air can cool it.
The numbers
- Oracle puts about 340 Atlas servers in its cloud this quarter.
- A neocloud company signed for Atlas and for the next server, Titan.
- Buyers want 90 to 120 days from a site with electrical supply to paid tokens.
What is not known
The source gives no test results for speed or cost against other processors. All claims come from the sales head of Positron. Positron plans to make a processor for Titan at the start of next year.
Insight: Memory shortage is the gap that Positron sells into
Kyle Dufresne of Dell estimates 20% more memory for the market next year, against about 200% more need. Positron does not use high-bandwidth memory, which is in short supply. Trend AI operates the Nvidia Nemotron model in customer data centers. Inference seems to move to small sites that customers control, and makers that use less of it get more control. Look for the date of Titan.
Sources
Posted