Compute
RaReCache lets a small model prefill the cache for a large model
The authors write that the large model keeps 95 to 99% of its precision when it recomputes only 30% of positions.
arxiv.orgClaimed, not confirmed
News category
Compute
49 stories, newest first.