提出无偏的KV缓存压缩方法，提升长文本处理能力
FAEDKV: Infinite-Window Fourier Transform for Unbiased KV Cache Compression
- 用无限窗傅里叶变换将KV缓存转到频域，均衡保留所有位置信息
- 在LongBench上性能优于现有方法最多22%，且无需模型重训练
- 适合需要长上下文记忆的部署场景，如文档问答与信息检索
大语言模型在长文本任务中的表现常受限于键值（KV）缓存带来的巨大内存开销和计算负担。现有压缩策略如令牌剔除和学习投影易产生偏差——或过度强调近期高注意力令牌，或反复损失早期上下文信息，且常需昂贵的模型重训练。我们提出FAEDKV（频率自适应无限窗KV缓存压缩框架），一种无需训练的新型压缩方法，确保信息无偏保留。FAEDKV通过提出的无限窗傅里叶变换（IWDFT）将KV缓存转换至频域，使所有令牌对压缩表示的贡献均等，有效同时保留早期与近期上下文。初步频域消融实验识别出各层关键频谱成分，支持分层定向压缩。在LongBench基准测试中，FAEDKV性能优于现有方法最高达22%。此外，该方法在针堆中找针任务中表现出更优、与位置无关的检索精度，优于基于压缩的方法。
原文摘要 · Abstract (English)
The efficacy of Large Language Models (LLMs) in long-context tasks is often hampered by the substantial memory footprint and computational demands of the Key-Value (KV) cache. Current compression strategies, including token eviction and learned projections, frequently lead to biased representations -- either by overemphasizing recent/high-attention tokens or by repeatedly degrading information from earlier context -- and may require costly model retraining. We present FAEDKV (Frequency-Adaptive Infinite-Window for KV cache), a novel, training-free KV cache compression framework that ensures unbiased information retention. FAEDKV operates by transforming the KV cache into the frequency domain using a proposed Infinite-Window Fourier Transform (IWDFT). This approach allows for the equalized contribution of all tokens to the compressed representation, effectively preserving both early and recent contextual information. A preliminary frequency ablation study identifies critical spectral components for layer-wise, targeted compression. Experiments on LongBench benchmark demonstrate FAEDKV's superiority over existing methods by up to 22\%. In addition, our method shows superior, position-agnostic retrieval accuracy on the Needle-In-A-Haystack task compared to compression based approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。