用任务表现反馈优化文档压缩，3%压缩比下反而提升准确率
Less Is More: Elevating RAG via Performance-Driven Context Compression
- 基于任务表现直接优化压缩策略，摆脱传统启发式方法
- 压缩至原始长度3%时，精确匹配率提升3.3个百分点
- 适合需要高效推理且保持高准确率的RAG应用
检索增强生成（RAG）已成为提升大模型知识更新时效性和事实准确性的重要范式。然而，大量检索文档显著增加输入长度，带来高昂计算成本。现有压缩方法常因依赖预设启发式规则而损害任务性能，这些规则无法确保压缩后的上下文对生成任务有利。为此，我们提出CORE-RAG框架，通过性能驱动的学习机制消除对代理启发式规则的依赖，直接以任务表现作为反馈信号，迭代优化压缩策略。在优化前，引入知识蒸馏阶段以初始化具备鲁棒性的压缩策略。大量实验表明，该方法优势显著：在3%的高压缩比下，不仅避免性能下降，平均精确匹配（EM）得分还较使用完整文档提升了3.3分。代码已公开于https://github.com/ziqiangcui/CORE-RAG-ICML26。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) has emerged as a promising paradigm for improving the timeliness of knowledge updates and the factual accuracy of large language models. However, incorporating a large volume of retrieved documents significantly increases input length, leading to prohibitive computational costs. Existing compression approaches often compromise task performance, primarily due to their reliance on predefined heuristics. These heuristics fail to ensure that the compressed context is conducive to the generation tasks. To address these limitations, we propose CORE-RAG, a novel framework for context compression in RAG systems. CORE eliminates reliance on proxy heuristics through a performance-driven learning framework, which directy utilizes task performance as a feedback signal to iteratively refine the compressor policy. Prior to this optimization process, we incorporate a knowledge distillation phase to initialize the compressor with a robust policy. Extensive experiments demonstrate the superiority of our approach. At a high compression ratio of 3%, CORE not only avoids performance degradation but also improves the average Exact Match (EM) score by 3.3 points compared to using full documents. Our code is available at https://github.com/ziqiangcui/CORE-RAG-ICML26.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。