通过分析模型内部表示,提升检索增强生成的忠实度。
Beyond Black-Box Interventions: Latent Probing for Faithful Retrieval-Augmented Generation
- 在隐空间中探测知识冲突,定位不一致信息
- 新框架使准确率与忠实度均显著提升
- 适合关注生成可信性的研究者和开发者
检索增强生成(RAG)系统常因无法保持上下文忠实性而生成与给定内容矛盾或未充分利用证据的回答。现有方法多采用外部干预,如特殊提示、解码校准或偏好优化,但因将大模型视为黑箱,缺乏对知识冲突发生时机与原因的可靠判断,导致方法脆弱、数据依赖强且忽略模型内部推理过程。本文突破黑箱干预,分析模型内部推理机制,发现冲突与一致的知识状态在隐空间中线性可分,且上下文噪声会系统性增加表示熵。基于此,提出ProbeRAG框架,包含三阶段:(i) 细粒度知识剪枝以过滤无关上下文,(ii) 隐空间冲突探测识别硬性冲突,(iii) 冲突感知注意力调节注意力头以实现忠实上下文融合。大量实验表明,ProbeRAG显著提升准确率与上下文忠实度。相关资源见https://github.com/LinfengGao/ProbeRAG。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) systems often fail to maintain contextual faithfulness, generating responses that conflict with the provided context or fail to fully leverage the provided evidence. Existing methods attempt to improve faithfulness through external interventions, such as specialized prompting, decoding-based calibration, or preference optimization. However, since these approaches treat the LLM as a black box, they lack a reliable mechanism to assess when and why knowledge conflicts occur. Consequently, they tend to be brittle, data-intensive, and agnostic to the model's internal reasoning process. In this paper, we move beyond black-box interventions to analyze the model's internal reasoning process. We discover that conflicting and aligned knowledge states are linearly separable in the model's latent space, and contextual noise systematically increases the entropy of these representations. Based on these findings, we propose ProbeRAG, a novel framework for faithful RAG that operates in three stages: (i) fine-grained knowledge pruning to filter irrelevant context, (ii) latent conflict probing to identify hard conflicts in the model's latent space, and (iii) conflict-aware attention to modulate attention heads toward faithful context integration. Extensive experiments demonstrate that ProbeRAG substantially improves both accuracy and contextual faithfulness. The related resources are available at https://github.com/LinfengGao/ProbeRAG.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。