The method of 4-bit compression changed a hidden trait in an AI model

Researchers put a hidden preference for pandas in the model Qwen2.5-7B-Instruct with subliminal learning (a model gets hidden traits from training data that seems unrelated). Compression to 8 bits did not change the trait in the model. At 4 bits, one method raised it and one method cut it to about 0.3 times its 16-bit value.

How it works

Quantization stores the weights of a model in 4 or 8 bits, and the model is small. The method HQQ raised the rate of panda answers from about 55% to 64%, and the controls stayed near 17%. The method NF4 cut the trait, but its rounding errors had the same dimension within 4%.

The cause

The authors say that the pattern of the rounding errors drives the change. When they shuffled the HQQ errors in each group, the trait moved between 0.2 and 1.6 times its 16-bit value.

What is not known

The test used one model and one harmless trait. The paper gives no result for a dangerous trait. The authors say that a 16-bit audit cannot bound risk, and that teams must test the weights that they deploy.

Insight: Compression steps can change what an audit checked

A new refinement step for low-bit quantization helps 6 methods and changes only the integer codes of the weights. That makes the weights different from the weights that an audit saw. This reads as a difference between the team that audits and the team that compresses, and the compressed file is the one that goes to users. Look for audit rules that name the deployed file.