让大模型动态决定何时查、怎么用外部知识,提升复杂任务表现
Dynamic and Parametric Retrieval-Augmented Generation
- 动态调整检索时机与内容,匹配生成过程中的实时需求
- 将知识注入从输入层升级到参数层,提高效率与效果
- 适合研究大模型知识增强或构建智能问答系统的读者
检索增强生成(RAG)已成为赋予大语言模型外部知识的基础范式,在信息检索与知识密集型应用中发挥关键作用。然而,传统RAG系统通常采用静态的‘检索-生成’流程,并依赖上下文中的知识注入,这在需要多跳推理、自适应信息访问和深度知识融合的复杂任务中表现不佳。为此,研究界正突破静态检索与上下文注入的局限。本教程深入探讨两大快速发展的互补方向:动态RAG与参数化RAG。动态RAG在生成过程中自适应地决定何时及检索何内容,实现对模型信息需求的实时响应;参数化RAG重新思考知识注入方式,从输入层转向参数层,以提升效率与效果。本文全面综述了这些新兴领域的最新进展,提供理论基础与实践洞见,助力推动RAG进一步发展。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) has become a foundational paradigm for equipping large language models (LLMs) with external knowledge, playing a critical role in information retrieval and knowledge-intensive applications. However, conventional RAG systems typically adopt a static retrieve-then-generate pipeline and rely on in-context knowledge injection, which can be suboptimal for complex tasks that require multihop reasoning, adaptive information access, and deeper integration of external knowledge. Motivated by these limitations, the research community has moved beyond static retrieval and in-context knowledge injection. Among the emerging directions, this tutorial delves into two rapidly growing and complementary research areas on RAG: Dynamic RAG and Parametric RAG. Dynamic RAG adaptively determines when and what to retrieve during the LLM's generation process, enabling real-time adaptation to the LLM's evolving information needs. Parametric RAG rethinks how retrieved knowledge should be injected into LLMs, transitioning from input-level to parameter-level knowledge injection for enhanced efficiency and effectiveness. This tutorial offers a comprehensive overview of recent advances in these emerging research areas. It also shares theoretical foundations and practical insights to support and inspire further research in RAG.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。