让检索增强生成系统记住过往查询,持续提升检索效果。
RAG without Forgetting: Continual Query-Infused Key Memory
- 通过正确性门控反馈,将每次查询的改进转化为持久的索引更新。
- 在13个领域测试中,推理速度不变,但检索与生成效果持续提升。
- 适合需要长期优化、追求零推理开销的RAG应用开发者。
检索增强生成(RAG)系统常通过查询时的适应性改进(如查询扩展和迭代检索)来提升鲁棒性。然而,这些方法本质上是无状态的:每次查询的调整仅临时存在,无法积累学习,且重复产生推理开销。索引端的方法(如键扩展)虽具持久性,但依赖离线预处理或启发式更新,与下游任务效用对齐不足,导致语义漂移和噪声累积。本文提出演化检索记忆(ERM),一种无需训练的框架,将查询时的收益转化为持久的检索改进。ERM通过正确性门控反馈更新检索索引,选择性地将原子扩展信号分配给其受益的文档键,并通过稳定、范数有界的更新逐步演化键。我们证明,在标准相似度函数下,查询扩展与键扩展等价,并证明了ERM选择性更新的收敛性,将最优查询扩展的收益摊销为稳定的索引,实现零推理开销。在BEIR与BRIGHT数据集上跨13个领域的实验表明,该方法在推理速度不变的前提下,显著提升了检索与生成性能,尤其在推理密集型任务中表现突出。
原文摘要 · Abstract (English)
Retrieval-augmented generation (RAG) systems commonly improve robustness via query-time adaptations such as query expansion and iterative retrieval. While effective, these approaches are inherently stateless: adaptations are recomputed for each query and discarded thereafter, precluding cumulative learning and repeatedly incurring inference-time cost. Index-side approaches like key expansion introduce persistence but rely on offline preprocessing or heuristic updates that are weakly aligned with downstream task utility, leading to semantic drift and noise accumulation. We propose Evolving Retrieval Memory (ERM), a training-free framework that transforms transient query-time gains into persistent retrieval improvements. ERM updates the retrieval index through correctness-gated feedback, selectively attributes atomic expansion signals to the document keys they benefit, and progressively evolves keys via stable, norm-bounded updates. We show that query and key expansion are theoretically equivalent under standard similarity functions and prove convergence of ERM's selective updates, amortizing optimal query expansion into a stable index with zero inference-time overhead. Experiments on BEIR and BRIGHT across 13 domains demonstrate consistent gains in retrieval and generation, particularly on reasoning-intensive tasks, at native retrieval speed.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。