让大模型答题更可信，解决知识与答案不一致的问题
Bridging the Knowledge-Prediction Gap in LLMs on Multiple-Choice Questions
- 通过分析隐藏层空间，发现知识与预测存在分离
- 提出KAPPA方法，使模型输出更符合内部知识
- 在多种题型和模型上有效，尤其适合考试类任务
尽管大型语言模型在各类任务中表现优异，但其可信度受限于行为的不可靠性，即输出行为与其内部知识不一致。特别是在多选题（MCQ）上，即使模型在隐层表示中编码了正确答案，仍常给出错误选择，暴露出知识与预测之间的错位。本文通过对隐藏表示的三步分析，研究并缓解这一知识-预测差距：首先量化了不同模型与数据集上该差距的普遍性和程度；其次从几何角度揭示残差流中存在独立的知识子空间与预测子空间；最后提出KAPPA，一种轻量级推理时干预方法，通过对齐残差流中的两个子空间来缩小差距。实验表明，该方法可有效减少多种多选题基准上的知识-预测差距，并推广至自由回答场景。
原文摘要 · Abstract (English)
While large language models (LLMs) perform strongly on diverse tasks, their trustworthiness is limited by erratic behavior that is unfaithful to their internal knowledge. In particular, LLMs often fail on multiple-choice questions (MCQs) even if they encode correct answers in their hidden representations, revealing a misalignment between internal knowledge and output behavior. We investigate and mitigate this knowledge-prediction gap on MCQs through a three-step analysis of hidden representations. First, we quantify the prevalence and magnitude of the gap across models and datasets. Second, we provide a geometric interpretation by identifying distinct knowledge and prediction subspaces in the residual stream. Third, we introduce KAPPA, a lightweight inference-time intervention that aligns the two subspaces within the residual stream to reduce the knowledge-prediction gap. Our results provide a geometric and interpretable explanation of the knowledge-prediction gap in LLMs. Furthermore, KAPPA effectively reduces the gap across diverse MCQ benchmarks and models, and generalizes to free-form settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。