提出新方法分析大模型生成解释时内外知识的多步互动
Multi-Step Knowledge Interaction Analysis via Rank-2 Subspace Disentanglement
- 用二维子空间解耦内/外知识贡献，突破传统单步一维限制
- 实验显示幻觉生成强烈依赖参数知识，真实回答则平衡内外知识
- 首次实现对长序列解释中知识交互的多步动态分析，适合可信AI研究者
自然语言解释（NLE）通过调用外部上下文知识（CK）和参数化知识（PK）来说明大语言模型（LLM）的决策过程。理解这两种知识源之间的交互对评估NLE的可信度至关重要，但相关动态仍缺乏深入研究。以往工作主要关注单步生成，且将PK与CK的交互建模为一维子空间中的二元选择，忽略了更丰富的交互模式及其在长序列生成中的演变过程，如互补或支持性知识。本文提出一种新的二维投影子空间，能更准确地解耦PK与CK的贡献，并首次实现对长序列NLE中知识交互的多步分析。在四个问答数据集和三个开源大模型上的实验表明，一维子空间难以表征多样化交互，而我们的二维方法能有效捕捉这些模式：支持性交互中表现出更强的PK对齐，冲突性交互中则表现为更强的CK对齐。多步分析还发现，幻觉生成显著偏向参数知识方向，而基于上下文忠实的生成则保持了PK与CK之间的更均衡对齐。
原文摘要 · Abstract (English)
Natural Language Explanations (NLEs) describe how Large Language Models (LLMs) make decisions by drawing on external Context Knowledge (CK) and Parametric Knowledge (PK). Understanding the interaction between these sources is key to assessing NLE grounding, yet these dynamics remain underexplored. Prior work has largely focused on i) single-step generation and ii) modeled PK--CK interaction as a binary choice within a rank-1 subspace. This approach overlooks richer interactions and how they unfold over longer generations, such as complementary or supportive knowledge. We propose a novel rank-2 projection subspace that disentangles PK and CK contributions more accurately and use it for the first multi-step analysis of knowledge interactions across longer NLE sequences. Experiments across four QA datasets and three open-weight LLMs demonstrate that rank-1 subspaces struggle to represent diverse interactions, whereas our rank-2 formulation captures them effectively, highlighting PK alignment for supportive interactions and CK alignment for conflicting ones. Our multi-step analysis reveals that hallucinated generations exhibit strong alignment with the PK direction, though context-faithful generations maintain a more balanced alignment between PK and CK.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。