提出GistPool方法，让大模型更好理解长文本提示。
Long Context In-Context Compression by Getting to the Gist of Gisting
- 用池化机制替代原有压缩方法，保持结构简单
- 在长上下文任务中性能远超原gisting方法
- 适合需要处理长输入的模型部署场景
长上下文处理对大模型应用至关重要，但现有方法常引入复杂架构，阻碍实际使用。无修改解码器结构的gisting方法因简洁和兼容性被看好，但在长上下文上表现不佳，即使压缩率极低也出现显著性能下降。令人意外的是，简单的平均池化基线始终优于gisting。分析表明，gisting存在信息流中断、容量不足及注意力无法聚焦子集等问题。基于理论洞察与大量实验，我们提出GistPool，保留gisting的简洁性，显著提升长上下文压缩任务性能。
原文摘要 · Abstract (English)
Long context processing is critical for the adoption of LLMs, but existing methods often introduce architectural complexity that hinders their practical adoption. Gisting, an in-context compression method with no architectural modification to the decoder transformer, is a promising approach due to its simplicity and compatibility with existing frameworks. While effective for short instructions, we demonstrate that gisting struggles with longer contexts, with significant performance drops even at minimal compression rates. Surprisingly, a simple average pooling baseline consistently outperforms gisting. We analyze the limitations of gisting, including information flow interruptions, capacity limitations and the inability to restrict its attention to subsets of the context. Motivated by theoretical insights into the performance gap between gisting and average pooling, and supported by extensive experimentation, we propose GistPool, a new in-context compression method. GistPool preserves the simplicity of gisting, while significantly boosting its performance on long context compression tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。