arXiv:2607.20538cs.LGcs.AI2026-07

通过学习新坐标系提升注意力缓存压缩质量,不改模型也能更好还原原结果。

Codec-Gauge: Learning Compression-Friendly Gauges for Transformer KV Caches

论文配图:Codec-Gauge: Learning Compression-Friendly Gauges for Transformer KV Caches
图 1 · 摘自论文原文
  • 在已有压缩算法上加一层正交变换,优化缓存数据布局
  • 3-6比特压缩下,平均降低44%的失真度(KL散度)
  • 适合做推理加速和模型部署的工程师,尤其关注压缩质量者

长上下文Transformer推理越来越依赖键值缓存的压缩或量化。先前研究表明,每个键/值向量的通道基底会影响固定后端对模型行为的保留程度。我们提出Codec-Gauge,一种后训练的缓存坐标层,可在现有压缩与量化后端基础上学习小型正交通道变换。其频谱分布目标结合了基于令牌-通道的DCT谱质心损失与平滑率代理,将KV能量集中在低频编码友好的布局中。通过实际测量字节数与滚动压缩历史评分评估压缩与解压效果。在六种模型、3、4和6比特/值条件下,所学坐标系相比原始坐标平均降低44.0%的zfp KL散度,并优于随机、哈达玛、DCT及PCA/KLT对照组。相同坐标系也提升了块均匀和KIVI式量化下的质量保持能力。270亿参数模型与长上下文任务提示的实验复现了质量提升趋势,串行存储与时间测量验证了压缩缓存路径的可行性。这些结果确立了缓存坐标几何作为提升压缩保真度的实用后训练变量,无需改变模型权重、注意力语义或后端编码规则。

原文摘要 · Abstract (English)

Long-context Transformer inference increasingly relies on KV-cache compression or quantization. Prior rotation and transform-coding results suggest that the channel basis of each key/value vector affects how faithfully a fixed backend preserves model behavior. We introduce Codec-Gauge, a post-training cache-coordinate layer that learns small orthogonal channel transforms around existing compression and quantization backends. Its frequency-distribution objective combines a token-channel DCT spectral-centroid loss with a smooth rate proxy to concentrate KV energy in low-frequency codec-facing layouts. We evaluate actual compression and decompression using measured bytes and rolling compressed-history scoring. Across six models at $3$, $4$, and $6$ bits/value, learned gauges reduce zfp KL divergence by $44.0\%$ on average relative to raw coordinates and outperform random, Hadamard, DCT, and PCA/KLT controls. The same gauges improve quality preservation for block-uniform and KIVI-style quantization. Experiments on a 27B model and long-context task prompts reproduce the quality trend, while serial storage and timing measurements validate the implemented compressed-cache paths. These results establish cache-coordinate geometry as a practical post-training variable for improving compression fidelity without changing model weights, attention semantics, or backend coding rules.

Transformer缓存压缩量化后训练优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。