用可学习的压缩标记，让大模型处理长文本时内存减少2到8倍。
Sentence-Anchored Gist Compression for Long-Context LLMs
- 通过训练模型自动生成压缩标记，替代冗余上下文。
- 在30亿参数LLaMA模型上实现8倍压缩率，性能不降。
- 适合需要高效处理长文档的场景，如法律、科研摘要。
本文研究利用可学习的压缩标记对大语言模型进行上下文压缩，以降低处理长序列时的内存与计算开销。实验表明，预训练大模型经微调后可在短上下文和长上下文基准测试中实现2至8倍的上下文压缩，且性能损失较小。在30亿参数的LLaMA模型上，该方法在压缩效果上优于其他技术，同时达到更高的压缩比。
原文摘要 · Abstract (English)
This work investigates context compression for Large Language Models (LLMs) using learned compression tokens to reduce the memory and computational demands of processing long sequences. We demonstrate that pre-trained LLMs can be fine-tuned to compress their context by factors of 2x to 8x without significant performance degradation, as evaluated on both short-context and long-context benchmarks. Furthermore, in experiments on a 3-billion-parameter LLaMA model, our method achieves results on par with alternative compression techniques while attaining higher compression ratios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。