通过智能合并检索内容,让大模型用更少的字获得更多信息。
Rethinking Retrieval-Augmentation as Synthesis: A Query-Aware Context Merging Approach
- 将检索结果按问题需求动态合成,而非简单筛选前k条。
- 在标准测试中提升13.7分F1和11.5分准确率。
- 适合需要高效利用上下文窗口的研究者与工程师。
检索增强生成(RAG)使大语言模型能够动态引入外部信息以扩展知识,但实际部署受限于模型有限的上下文窗口,需在信息量与令牌消耗间权衡。现有方法采用‘检索-选择’策略,通常仅保留相关性最高的前k个文本块,但该方式会丢弃长尾中关键的连接性证据,同时浪费令牌在语义重复的高分块上。本文将检索增强重新视为最大化信息密度的动态优化问题,提出MergeRAG框架,从静态过滤转向查询感知的合成。MergeRAG通过评分代理使用双路径机制重构上下文:对称合并整合弱信号以恢复丢失的桥接证据;非对称合并利用熵引导锚定消除冗余而不损失语义完整性。进一步提出分层并行合并策略,在减少信息损失的同时最大化计算并行度。在多个标准基准上的大量实验表明,MergeRAG显著优于现有SOTA RAG基线,F1得分提升最高达13.7点,精确匹配(EM)提升11.5点。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) enables Large Language Models (LLMs) to extend their existing knowledge by dynamically incorporating external information. However, practical deployment is fundamentally constrained by the LLM's finite context window, forcing a trade-off between information sufficiency and token consumption. Standard pipelines address this via a retrieve-then-select strategy, typically retaining only the top-k chunks based on relevance. Nevertheless, this approach is suboptimal: it inherently truncates critical bridging evidence located in the long tail of the relevance distribution, while simultaneously wasting the token budget on semantically redundant high-ranking chunks. In this paper, we rethink retrieval-augmentation as a dynamic optimization problem aimed at maximizing information density. We propose MergeRAG, a novel framework that shifts the paradigm from static filtering to query-aware synthesis. MergeRAG employs a scoring agent to restructure retrieved contexts through a dual-pathway mechanism: 1) Symmetric Merging, which consolidates weak signals to recover lost bridging evidence; 2) Asymmetric Merging, which utilizes entropy-guided anchoring to eliminate redundancy without sacrificing semantic integrity. We further introduce a Hierarchical Parallel Merging strategy that mitigates information loss while maximizing computational parallelism. Extensive experiments on standard benchmarks demonstrate that MergeRAG significantly outperforms state-of-the-art RAG baselines, achieving up to 13.7 points improvement in F1 score and 11.5 points in Exact Match (EM), respectively.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。