A frozen model cannot use a KV cache from a different model
A reader that keeps its cache and uses the cache of the sender also gives more correct results.
Researchers test if a frozen model can read the KV cache of a different model. The weights do not change. A KV cache is data that a model keeps while it reads text. 8 substitution methods all stay at the baseline. The authors say that a frozen transformer reads only caches that are near the cache that it makes. In a different setup, the reader keeps the cache that it made. It also uses the cache of the sender. It gives more correct results than the baseline.
Sources
Posted
