1
Speculative KV coding: losslessly compressing KV cache by up to ~4×
Lossless compression of a target model's KV cache by up to 4×, using a cheaper predictor model to drive an arithmetic coder.
Fergus Finn
A blog about LLM inference
- Popular posts
- 3
- Total points
- 569
- HN comments
- 105
- Rank
- #4,420 personal
Visit fergusfinn.com → RSS feedOn Hacker News Jun 2026 – Jun 2026
Hardware & Electronics 2AI & Machine Learning 2Systems & Performance 1Databases & Data 1