揭示大模型答题时的决策轨迹，区分稳定与不稳定的正确答案。
The Shape of Wisdom: Decision Trajectories in Language Models
- 通过分析答案置信度变化轨迹，量化决策过程
- 9000次实验显示多数正确答案不稳固，易受干扰
- 注意力机制对正确性有推动作用，而前馈网络影响较小
语言模型并非在输出层直接选择答案。在 Qwen2.5-7B-Instruct、Llama-3.1-8B-Instruct 与 Mistral-7B-Instruct-v0.3 上开展的 9,000 条轨迹 MMLU 研究发现，答案得分在模型深度中以结构化方式演变。我们用三个量描述每条轨迹：当前答案置信度差值、下一层该差值的变化量、距离决策翻转的距离。主要发现是：正确性与稳定性不同，最大群体为不稳定-正确，而非稳定-正确。进一步追踪子集发现，在稳定-正确的案例中，平均注意力标量指向正确方向，而平均 MLP 标量则不然；删除支持性文本会削弱置信度差值，删除干扰类文本则有助于提升。结果并非完整电路解释，但提供了一种可复现的方法，用于识别哪些答案已确定、哪些仍脆弱，以及哪些因素真正驱动了决策变化。
原文摘要 · Abstract (English)
Language models do not simply choose an answer at the output layer. In a 9,000-trajectory MMLU study across Qwen2.5-7B-Instruct, Llama-3.1-8B-Instruct, and Mistral-7B-Instruct-v0.3, the score of the answer moves across depth in structured ways. We describe each trajectory with three quantities: the current answer margin, the next-layer change in that margin, and the distance from a decision flip. The main empirical picture is that correctness and stability are different: the largest group is unstable-correct, not stable-correct. A traced subset then asks what moves the margin. In stable-correct cases, the average attention scalar points in the correct direction, while the average MLP scalar does not; span deletion shows that removing answer-supporting text hurts the margin and removing distractor-like text helps it. The result is not a full circuit explanation. It is a reproducible way to see which answers are settled, which remain fragile, and which measured sources move them.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。