让检索增强生成系统学会动态记忆,越用越快越准。
GAM-RAG: Gain-Adaptive Memory for Evolving Retrieval in Retrieval-Augmented Generation
- 用动态记忆替代静态索引,根据查询重复性自动更新检索路径。
- 在5轮记忆下性能提升8.19%,推理成本降低61%。
- 适合需要长期优化的对话、问答等持续交互场景。
检索增强生成(RAG)通过外部证据为大语言模型提供依据,但多数实现依赖构建后不再变化的静态索引。相似或相关查询反复进行多跳遍历,导致延迟和计算开销增加。受认知神经科学中基于模式的学习启发,我们提出GAM-RAG——一种无需训练的框架,能积累重复或相关查询的检索经验,并随时间更新检索记忆。GAM-RAG构建轻量级、无关系的分层索引,其链接反映潜在共现而非固定语义关系。推理时,成功的检索经历提供句级反馈,更新句子记忆,使未来类似推理类型更易激活相关证据。为平衡噪声反馈下的稳定与适应性,引入基于卡尔曼滤波思想的不确定性感知增益规则,对可靠新信号快速更新,对稳定或噪声记忆保守修正。我们提供了更新动力学的理论分析,并实证表明,GAM-RAG相比最强基线平均性能提升3.95%,在5轮记忆下提升8.19%,同时推理成本降低61%。代码与数据集见:https://anonymous.4open.science/r/GAM_RAG-2EF6。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) grounds large language models with external evidence, but many implementations rely on pre-built indices that remain static after construction. Related queries therefore repeat similar multi-hop traversal, increasing latency and compute. Motivated by schema-based learning in cognitive neuroscience, we propose GAM-RAG, a training-free framework that accumulates retrieval experience from recurring or related queries and updates retrieval memory over time. GAM-RAG builds a lightweight, relation-free hierarchical index whose links capture potential co-occurrence rather than fixed semantic relations. During inference, successful retrieval episodes provide sentence-level feedback, updating sentence memories so evidence useful for similar reasoning types becomes easier to activate later. To balance stability and adaptability under noisy feedback, we introduce an uncertainty-aware, Kalman-inspired gain rule that jointly updates memory states and perplexity-based uncertainty estimates. It applies fast updates for reliable novel signals and conservative refinement for stable or noisy memories. We provide a theoretical analysis of the update dynamics, and empirically show that GAM-RAG improves average performance by 3.95% over the strongest baseline and by 8.19% with 5-turn memory, while reducing inference cost by 61%. Our code and datasets are available at: https://anonymous.4open.science/r/GAM_RAG-2EF6.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。