用混合格式压缩注意力键值缓存，不丢精度还提速。
Minima-KV: Retention-Preserving KV Cache Compression with Mixed-Format Paged Attention
- 新缓存结构：锚点页保精度用FP8，旧页压缩成TQ3格式
- 16K上下文任务性能仅降0.4个百分点，64K仍保持稳定
- 无需密集影子缓存，适合长文本生成场景
键值（KV）缓存是长上下文大模型服务中的主要容量与带宽瓶颈。本文提出Minima-KV，一种保留关键信息的混合格式分页注意力层级结构。最近和受保护的锚点页以FP8格式保留，较老的非锚点页则压缩为打包的TQ3格式；每个活跃请求页均可直接寻址。特定格式的内核计算部分注意力状态，并通过全局归一化的在线Softmax合并，实现无需缓存级密集影子的异构解码。在单块96GB NVIDIA RTX PRO 6000 Blackwell GPU上，针对Qwen3.6-27B的独立配置，在部署计费中每活跃令牌仅需18.3 KiB注意力KV，相比BF16压缩3.50倍，相比FP8压缩1.75倍。材料化质量测试在16K RULER Needle-in-a-Haystack任务上与密集对照组持平。在503题的LongBench v2测试集上，16K、32K、64K上下文的性能差异分别为-0.80、-0.60、-0.40百分点。单独一对直接解码基准测试中，两个59,008令牌请求实现3.625倍活跃KV压缩，吞吐率为控制组的0.9821倍，16层全注意力均无回退，且未保留密集影子。这些结果验证了无需驱逐活跃请求页即可实用压缩长上下文状态的混合格式路径。
原文摘要 · Abstract (English)
The key-value (KV) cache is a primary capacity and bandwidth bottleneck in long-context LLM serving. We present Minima-KV, a retention-preserving hierarchy for mixed-format paged attention. Recent and protected Anchor pages remain in FP8, while older non-anchor pages move to packed TQ3; every live-request page remains addressable. Format-specific kernels compute partial attention states and combine them through a globally normalized online-softmax merge, enabling direct heterogeneous decode without a cache-sized dense shadow. Across separate, configuration-bound Qwen3.6-27B profiles on a single 96-GB NVIDIA RTX PRO 6000 Blackwell GPU, deployment accounting reports 18.3 KiB of attention KV per live token, corresponding to 3.50x compression relative to BF16 and 1.75x relative to FP8. A materializing quality profile matches its dense control on 16K RULER needle-in-a-haystack tasks. On the same 503-question LongBench v2 set, measured deltas are -0.80, -0.60, and -0.40 percentage points at 16K, 32K, and 64K. A separate single-pair direct-decode canary with two 59,008-token requests measures 3.625x active-KV compression and 0.9821x throughput relative to its control, routes all 16 full-attention layers without fallback, and retains no dense shadow. These results establish a practical mixed-format path for compressing long-context state without evicting live-request KV pages.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。