用单个标记实时判断推理对错，大幅减少计算量。
One-Token Verification for Reasoning Correctness Estimation
- 引入可学习标记，在生成时单次前向传播判断推理正确性。
- 在数学推理任务上优于现有验证方法，最多节省90%生成token。
- 适合需要高效推理验证的场景，如实时问答与自动评估。
大语言模型在复杂推理任务（如数学解题）中取得显著进展。常用策略是并行生成多条推理路径，再通过多数投票或最佳N选一等聚合方式决定结果。然而仍存在两大挑战：多样本解码带来显著推理延迟，尤其在长文本输出时；且缺乏可靠机制来评估单条推理路径的正确性。为此，本文提出一次性验证（One-Token Verification, OTV），一种在生成过程中仅需一次前向传播即可估计推理正确性的计算方法。OTV通过可学习标记激活，利用低秩适配技术探查键值缓存中的内部推理信号，实现任意生成阶段的细粒度正确性判断，且不干扰主推理流程。在数学推理基准测试中，OTV表现持续优于现有验证器。此外，借助正确性引导的提前终止，OTV可将令牌使用量减少高达90%，优先选择更短、更可靠的解法。
原文摘要 · Abstract (English)
Recent breakthroughs in large language models (LLMs) have led to notable successes in complex reasoning tasks, such as mathematical problem solving. A common strategy for improving performance is parallel thinking, in which multiple reasoning traces are generated and the final prediction is made using aggregation schemes like majority voting or best-of-$N$ decoding. However, two key challenges persist. First, multi-sample decoding incurs substantial inference latency, especially for long-form outputs. Second, effective mechanisms for reliably assessing the correctness of individual reasoning traces are still limited. To address these challenges, we introduce One-Token Verification (OTV), a computational method that estimates reasoning correctness in a single forward pass during generation. OTV is activated by a learnable token and integrated into the LLM via low-rank adaptation to probe internal reasoning signals through the key-value cache, supporting token-level correctness estimation at any stage of generation without disrupting primary reasoning. Experiments on mathematical reasoning benchmarks demonstrate that OTV consistently surpasses existing verifiers. Additionally, OTV reduces token usage by up to $90\%$ through correctness-guided early termination, prioritizing shorter, more reliable solutions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。