What matters in AI.

Subscribe

LookAtMyQuant cuts Inkling 2 Mini from 18.89 GB to 5.38 GB

The authors report that the 5.38 GB version beats a 9.14 GB rival version on two tests.

LookAtMyQuant is a free tool, with an MIT license, that cuts the GB of AI models. It measures how much each saved part of a model damages the output when it is compressed. It then selects a code for each part, and the version fits in a set number of GB. The authors used it on Inkling-2-Mini from Thinking Machines and report the results.

How it works

The tool gives more bits to the parts that are most important. It then trains the small version for 4,000 steps to copy the initial model. A small draft model adds speed.

The numbers

  • The 5.38 GB version beats a 9.14 GB Unsloth version on GSM8K (math) and HumanEval (code).
  • The 8.87 GB version matches the initial model on GSM8K.
  • On an Apple M5 Max, the 5.4 GB version makes a median of 125.2 tokens each second.

What is not known

The authors ran all tests. They used 200 math problems and 164 code problems. The source gives no accurate scores for the 5.38 GB version and no external test.

Insight: Open weights let outsiders shrink a model more

Thinking Machines says Inkling 2 Mini operates on a laptop with about 20 GB of memory. An external project at this time makes it 5.38 GB. PrismML made a 27B Qwen model below 6 GB. Open weights seem to start a race of external compressors, and the model maker gets users at no cost. Look for more such versions.

Sources

  1. Inkling-2-Mini in 5.4 GB Memorygithub.com
AI MATTER · NEWS · AI MATTER · NEWS ·21 NOV2026

Posted

Tags

Learn the terms in this story

Guides for this story

More in Models

All Models news