用注意力机制动态压缩检索内容，提升大模型生成准确性与速度。
AttnComp: Attention-Guided Adaptive Context Compression for Retrieval-Augmented Generation
- 基于注意力权重筛选关键信息，自适应调整压缩比例。
- 压缩率高且延迟低，在多个数据集上准确率超越基线。
- 可估算生成置信度，适合对可靠性要求高的应用。
检索增强生成通过引入外部上下文提升大语言模型的事实准确性，但常受无关内容干扰。上下文压缩通过在生成前过滤冗余信息缓解此问题。然而，现有方法难以根据不同上下文自适应调整压缩率，同时兼顾低延迟和多文档信息整合。为此，我们提出AttnComp——一种自适应、高效且上下文感知的压缩框架。该方法利用大模型的注意力机制识别相关性，采用Top-P压缩算法保留累积注意力权重超过预设阈值的最小文档集合。此外，AttnComp通过评估检索内容的整体相关性估计生成置信度，帮助用户判断输出可靠性。实验表明，AttnComp优于现有压缩方法及未压缩基线，在实现显著压缩率的同时保持更高准确率和更低延迟。
原文摘要 · Abstract (English)
Retrieval-augmented generation improves the factual accuracy of Large Language Models (LLMs) by incorporating external context, but often suffers from irrelevant retrieved content that hinders effectiveness. Context compression addresses this issue by filtering out irrelevant information from context before LLM generation. However, existing methods struggle to adaptively adjust compression rates for different context, maintain low latency and integrate information across multiple documents. To overcome these limitations, We introduce AttnComp, an adaptive, efficient and context-aware compression framework. By leveraging the attention mechanism of LLMs to identify relevant information, AttnComp employs a Top-P compression algorithm to retain the minimal set of documents whose cumulative attention weights exceeds a predefined threshold. In addition to compression, AttnComp estimates response confidence by assessing the overall relevance of the retrieved content, enabling users to gauge response reliability. Experiments demonstrate that AttnComp outperforms existing compression methods and uncompressed baselines, achieving higher accuracy with substantial compression rates and lower latency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。