发现并抑制大模型回答突变的关键注意力头，提升可信度。
Interpreting and Mitigating Unwanted Uncertainty in LLMs
- 通过模拟重提示场景，定位导致答案突变的非检索注意力头。
- 屏蔽特定注意力头后，答案翻转率降低最多15%，且不引发新错误。
- 适合关注模型可靠性与可解释性的研究人员和应用开发者。
尽管大型语言模型表现出色，但其存在不希望出现的不确定性——即在重新提示时，将原本正确的答案变为错误。这种行为损害信任，在高风险领域带来严重风险。本文研究该现象的机制，采用针堆找针检索框架，并结合翻转式重评估提示，模拟真实场景下的答案突变。发现检索头并非主要责任方；相反，一小部分非检索注意力头在不确定语境中过度关注误导性标记。屏蔽这些头可显著改善表现，使翻转行为减少高达15%，且未引入不连贯或过度修正。但在下游任务测试中，仍观察到翻转行为的权衡。研究成果推动机制可解释性发展，提出一种简单有效的缓解不确定性失效模式的方法。
原文摘要 · Abstract (English)
Despite their impressive capabilities, Large Language Models (LLMs) exhibit unwanted uncertainty, a phenomenon where a model changes a previously correct answer into an incorrect one when re-prompted. This behavior undermines trust and poses serious risks in high-stakes domains. In this work, we investigate the mechanisms that drive this phenomenon. We adapt the Needle-in-a-Haystack retrieval framework and integrate a Flip-style re-evaluation prompt to simulate realistic answer-flipping scenarios. We find that retrieval heads are not primarily responsible for avoiding uncertainty. Instead, we identify a small set of non-retrieval attention heads that disproportionately attend to misleading tokens in uncertain contexts. Masking these heads yields significant improvements, reducing flip behavior by up to 15% without introducing incoherence or overcorrection. However, when tested for downstream tasks, we observe trade-offs with flip behavior. Our findings contribute to the growing field of mechanistic interpretability and present a simple yet effective technique for mitigating uncertainty-driven failure modes in LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。