Anthropic operated a 26 billion parameter model on 15 TV boxes
The team wrote that a copy of such a model is not easy to stop, because the cluster has no one point to stop.
Claimed, not confirmed
On October 12, the Frontier Red Team of Anthropic wrote a report on a test. Researchers William Zhou and Daniel Freeman divided a model of 26 billion parameters across 15 Android TV boxes. Each box was about $55 and had 4GB of memory. The cluster made 0.39 tokens each second. The team wrote that such a system is not easy to stop, because it has no one point to stop.
The set-up
The researchers used Gemma4-26B-A4B, a model from Google.
Each box was about $55 and had 4GB of memory. The model does not fit in the memory of one box. Each box had 2 layers of the model.
The result
The cluster made 0.39 tokens each second. The speed of a GPU cluster is high. The test shows that the model can operate on hardware that a person can get.
Why it is hard to stop
The test is part of the tests of Anthropic on autonomous replication. These tests check if a model can make a copy of the model and move it to other hardware.
The team wrote that such a copy is not easy to stop, because the cluster has no one point to stop. If you remove one box, that part of the model is not in the system until a different device replaces the box.
The researchers wrote that it is slow to add more boxes. The copy moves to more boxes in days, not hours.
The limits
Models with many more parameters are different. A speed of 0.39 tokens each second is too low for a system for customers.
Sources
Posted
