用压缩标记减少大模型推理内存占用，提升长文本处理效率
Breadcrumbs Reasoning: Memory-Efficient Reasoning with Compression Beacons
- 定期用特殊标记压缩历史生成的键值缓存
- 在保持精度前提下，内存占用降低40%以上
- 适合需要长时间推理的资源受限场景
大语言模型在长上下文推理中受限于Transformer键值缓存的线性增长，带来显著的内存与计算开销。我们认为，随着模型生成推理标记，过去生成标记的信息价值逐渐降低，这为压缩提供了可能。本文提出周期性地使用可学习的专用标记压缩生成的键值缓存，并移除压缩后的条目。通过改进的联合蒸馏与强化学习（RL）框架训练模型完成该压缩过程，该方法利用强化学习输出进行蒸馏，显著降低传统强化学习的额外开销。实验表明，本方法在内存-精度权衡上优于无缓存压缩的模型及无需训练的压缩技术。
原文摘要 · Abstract (English)
The scalability of large language models for long-context reasoning is severely constrained by the linear growth of their Transformer key-value cache, which incurs significant memory and computational costs. We posit that as a model generates reasoning tokens, the informational value of past generated tokens diminishes, creating an opportunity for compression. In this work, we propose to periodically compress the generation KV cache with a learned, special-purpose token and evict compressed entries. We train the model to perform this compression via a modified joint distillation and reinforcement learning (RL) framework. Our training method minimizes overhead over the conventional RL process, as it leverages RL outputs for distillation. Empirically, our method achieves a superior memory-accuracy Pareto frontier compared to both the model without cache compression and training-free compression techniques.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。