探究大模型如何在内部与外部知识间做选择
Probing Language Models on Their Knowledge Source
- 设计可控提示，测试模型在冲突知识下的选择机制
- 中层激活值能准确预测模型使用哪种知识源
- 适用于研究模型决策逻辑或提升知识可靠性的人
大型语言模型（LLMs）常面临其内部参数化知识（PK）与推理时提供的外部上下文知识（CK）之间的冲突。理解模型如何在两者间权衡仍具挑战。本文提出一种新型探针框架，通过设计与模型内部知识矛盾的控制性提示，揭示特定模型激活值可指示知识来源。我们在不同规模的多种LLMs上评估该框架，发现中层激活值，尤其是与输入关系相关的部分，在预测知识源选择中起关键作用，为构建能有效处理知识冲突的更可靠模型铺平道路。
原文摘要 · Abstract (English)
Large Language Models (LLMs) often encounter conflicts between their learned, internal (parametric knowledge, PK) and external knowledge provided during inference (contextual knowledge, CK). Understanding how LLMs models prioritize one knowledge source over the other remains a challenge. In this paper, we propose a novel probing framework to explore the mechanisms governing the selection between PK and CK in LLMs. Using controlled prompts designed to contradict the model's PK, we demonstrate that specific model activations are indicative of the knowledge source employed. We evaluate this framework on various LLMs of different sizes and demonstrate that mid-layer activations, particularly those related to relations in the input, are crucial in predicting knowledge source selection, paving the way for more reliable models capable of handling knowledge conflicts effectively.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。