模型提前知道答案却拖延输出，新方法可提前终止并提升准确率。
The Detection-Extraction Gap: Models Know the Answer Before They Can Say It
- 利用自由生成检测答案出现时机，再强制提取，避免冗余计算。
- 70%~78%的推理过程可被提前截断，准确率提升1%~5个百分点。
- 适合需要高效推理的场景，尤其对思考型模型效果显著。
现代推理模型在答案已确定后仍持续生成大量文本。在五种模型配置、两个模型家族和三个基准测试中，52%~88%的思维链（chain-of-thought）token是在答案从部分前缀即可恢复之后产生的。这种现象称为‘检测-提取差距’：从早期前缀的自由延续能恢复正确答案，但强制提取在42%的情况下失败。答案虽可从模型状态中恢复，但提示条件解码无法有效提取。我们通过总变差界形式化该不匹配，量化了后缀引入的分布偏移。基于此不对称性，提出黑盒自适应早退（BAEE），利用自由延续实现检测与提取，使序列生成减少70%~78%，所有模型准确率提升1%~5个百分点。对思考模式模型，早退防止答案被后续内容覆盖，最高提升5.8个百分点；成本优化版本实现68%~73%减少，中位仅需9次API调用。代码见https://github.com/EdWangLoDaSc/know2say。
原文摘要 · Abstract (English)
Modern reasoning models continue generating long after the answer is already determined. Across five model configurations, two families, and three benchmarks, we find that 52--88% of chain-of-thought tokens are produced after the answer is recoverable from a partial prefix. This post-commitment generation reveals a structural phenomenon: the detection-extraction gap. Free continuations from early prefixes recover the correct answer even at 10% of the trace, while forced extraction fails on 42% of these cases. The answer is recoverable from the model state, yet prompt-conditioned decoding fails to extract it. We formalize this mismatch via a total-variation bound between free and forced continuation distributions, yielding quantitative estimates of suffix-induced shift. Exploiting this asymmetry, we propose Black-box Adaptive Early Exit (BAEE), which uses free continuations for both detection and extraction, truncating 70--78% of serial generation while improving accuracy by 1--5pp across all models. For thinking-mode models, early exit prevents post-commitment overwriting, yielding gains of up to 5.8pp; a cost-optimized variant achieves 68--73% reduction at a median of 9 API calls. Code is available at https://github.com/EdWangLoDaSc/know2say.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。