揭秘大模型如何融合内部知识与外部检索信息,提升问答可靠性。
Unveiling Knowledge Utilization Mechanisms in LLM-based Retrieval-Augmented Generation
- 分四个阶段解析大模型处理内外知识的流程
- 发现检索相关性决定知识流动路径,关键神经元可被精准调控
- 适合研究模型可解释性、改进智能助手的开发者
由于大语言模型(LLM)的参数化知识存在固有局限,检索增强生成(RAG)被广泛用于扩展其知识范围。尽管RAG在开放域问答等知识密集型任务中表现优异,其内在知识利用机制仍不清晰。本文系统研究了LLM在RAG场景下整合内部与外部知识的机制。通过宏观层面的知识流分析和微观层面模块功能探究,我们识别出知识利用过程的四个阶段:知识精炼、知识唤醒、知识表达和知识竞争。结果表明,检索段落的相关性引导知识在各阶段的流动。在模块层面,提出知识激活概率熵(KAPE)方法,定位与内/外知识相关的神经元,并通过选择性关闭实现对知识源依赖性的定向调整。此外,发现多头注意力与前馈网络在知识构建中具有互补作用。这些发现为提升检索增强型大模型的可解释性与可靠性提供了基础,推动知识密集领域生成系统的鲁棒性与透明度发展。
原文摘要 · Abstract (English)
Considering the inherent limitations of parametric knowledge in large language models (LLMs), retrieval-augmented generation (RAG) is widely employed to expand their knowledge scope. Since RAG has shown promise in knowledge-intensive tasks like open-domain question answering, its broader application to complex tasks and intelligent assistants has further advanced its utility. Despite this progress, the underlying knowledge utilization mechanisms of LLM-based RAG remain underexplored. In this paper, we present a systematic investigation of the intrinsic mechanisms by which LLMs integrate internal (parametric) and external (retrieved) knowledge in RAG scenarios. Specially, we employ knowledge stream analysis at the macroscopic level, and investigate the function of individual modules at the microscopic level. Drawing on knowledge streaming analyses, we decompose the knowledge utilization process into four distinct stages within LLM layers: knowledge refinement, knowledge elicitation, knowledge expression, and knowledge contestation. We further demonstrate that the relevance of passages guides the streaming of knowledge through these stages. At the module level, we introduce a new method, knowledge activation probability entropy (KAPE) for neuron identification associated with either internal or external knowledge. By selectively deactivating these neurons, we achieve targeted shifts in the LLM's reliance on one knowledge source over the other. Moreover, we discern complementary roles for multi-head attention and multi-layer perceptron layers during knowledge formation. These insights offer a foundation for improving interpretability and reliability in retrieval-augmented LLMs, paving the way for more robust and transparent generative solutions in knowledge-intensive domains.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。