模型能自检对错，用隐藏状态判断推理是否该停止。
Reasoning Models Know When They're Right: Probing Hidden States for Self-Verification
- 通过探测隐藏状态，识别中间答案的正确性。
- 验证准确率高，还能提前预测未来答案正确性。
- 用自检机制减少24%推理词数，不降低性能。
推理模型在数学和逻辑任务上表现优异，得益于其推理过程中的搜索能力。然而，它们仍存在过度推理问题，即使已得出正确答案也继续执行不必要的步骤。本文研究推理模型是否通过探测隐藏状态来编码答案正确性的信息。结果表明，该探测器能以高准确率验证中间答案，并生成高度校准的置信度分数。此外，模型隐藏状态还编码未来答案的正确性，可在中间答案形成前提前预测其正确性。我们进一步将探测器用作验证器，在推理中决定是否在中间步骤退出，使推理词数减少24%且不降低性能。这些发现证实推理模型确实编码了正确性概念，但未能有效利用，揭示了显著提升效率的潜力。
原文摘要 · Abstract (English)
Reasoning models have achieved remarkable performance on tasks like math and logical reasoning thanks to their ability to search during reasoning. However, they still suffer from overthinking, often performing unnecessary reasoning steps even after reaching the correct answer. This raises the question: can models evaluate the correctness of their intermediate answers during reasoning? In this work, we study whether reasoning models encode information about answer correctness through probing the model's hidden states. The resulting probe can verify intermediate answers with high accuracy and produces highly calibrated scores. Additionally, we find models' hidden states encode correctness of future answers, enabling early prediction of the correctness before the intermediate answer is fully formulated. We then use the probe as a verifier to decide whether to exit reasoning at intermediate answers during inference, reducing the number of inference tokens by 24\% without compromising performance. These findings confirm that reasoning models do encode a notion of correctness yet fail to exploit it, revealing substantial untapped potential to enhance their efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。