提出查询感知的软压缩框架,让检索生成更高效准确
Rethinking Soft Compression in Retrieval-Augmented Generation: A Query-Conditioned Selector Perspective
- 用查询条件选择器替代全量压缩,动态筛选相关信息
- 在多个数据集上超越现有压缩方法,计算量减少33.8%~84.6%
- 适合需要低延迟、高精度的在线问答和知识密集型任务
检索增强生成(RAG)通过外部知识提升大语言模型性能,但受限于上下文过长和冗余检索。现有软压缩方法采用类似自编码器的全量压缩,强制编码器压缩全部文档内容,忽略查询相关性,导致性能下降。本文分析发现该范式存在两大问题:一是全压缩与生成行为不兼容;二是非必要压缩稀释了任务相关信息密度。为此,提出SeleCom框架,将编码器重构为查询感知的信息选择器,仅保留与查询相关的部分。该选择器为解码器结构,基于大规模、多样化且难度分级的合成QA数据集,采用课程学习训练。实验表明,SeleCom显著优于现有软压缩方法,在保持或超越非压缩基线性能的同时,计算量与延迟降低33.8%~84.6%。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) effectively grounds Large Language Models (LLMs) with external knowledge and is widely applied to Web-related tasks. However, its scalability is hindered by excessive context length and redundant retrievals. Recent research on soft context compression aims to address this by encoding long documents into compact embeddings, yet they often underperform non-compressed RAG due to their reliance on auto-encoder-like full-compression that forces the encoder to compress all document information regardless of relevance to the input query. In this work, we conduct an analysis on this paradigm and reveal two fundamental limitations: (I) Infeasibility, full-compression conflicts with the LLM's downstream generation behavior; and (II) Non-necessity: full-compression is unnecessary and dilutes task-relevant information density. Motivated by these insights, we introduce SeleCom, a selector-based soft compression framework for RAG that redefines the encoder's role as query-conditioned information selector. The selector is decoder-only and is trained with a massive, diverse and difficulty-graded synthetic QA dataset with curriculum learning. Extensive experiments show that SeleCom significantly outperforms existing soft compression approaches and achieves competitive or superior performance to non-compression baselines, while reducing computation and latency by 33.8%~84.6%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。