arXiv:2605.20075cs.CLcs.AI2026-05

让大模型先出答案再反思,提升推理效率与准确率

CopT: Contrastive On-Policy Thinking with Continuous Spaces for General and Agentic Reasoning

论文配图:CopT: Contrastive On-Policy Thinking with Continuous Spaces for General and Agentic Reasoning
图 1 · 摘自论文原文
  • 先生成草稿答案,再基于答案进行针对性反思
  • 通过对比嵌入与离散输入的差异,判断答案可靠性
  • 无需额外训练,数学、编程等任务准确率最高提升23%

链式思考(CoT)是激发大语言模型推理能力的标准方法,但传统CoT将思考视为回答的前提,可能导致延迟响应并产生不必要的令牌开销,尤其当模型已能快速识别答案时。本文提出CopT,一种重构的推理流程:先生成草稿答案,再基于该答案进行在策略反思与修正。为评估草稿答案的可信度,CopT将连续嵌入重新定义为推理时的对比验证器,通过对比离散标记输入与连续嵌入输入下对相同生成标记的支持程度,获得序列级反向KL估计器以衡量答案可靠性。分析表明,在特定假设下,该估计量期望值等于未解潜状态与输出答案标记之间的互信息,从而捕捉与答案相关的不确定性而非任意潜状态不确定性。当答案不可靠时,系统执行进一步在策略思考,利用第二个KL估计器动态控制草稿可见性,保留有用部分信息同时降低被误导风险。在数学、编码及智能体推理任务中,CopT将峰值准确率提升最高达23%,令牌消耗减少最高达57%,且无需额外训练。代码已开源。

原文摘要 · Abstract (English)

Chain-of-thought (CoT) is a standard approach for eliciting reasoning capabilities from large language models (LLMs). However, the common CoT paradigm treats thinking as a prerequisite for answering, which can delay access to plausible answers and incur unnecessary token costs even when the model is able to identify an answer before extended thinking, a behavior known as performative reasoning. In this paper, we introduce CopT, a reformulated reasoning pipeline that reverses the usual order of thinking and answering. Instead of thinking before answering, CopT first elicits a draft answer and then invokes subsequent on-policy thinking conditioned on its own draft answer for reflection and correction. To assess whether the draft answer should be trusted, CopT recasts continuous embeddings as inference-time contrastive verifiers. Specifically, it contrasts the model's support for the same generated tokens under discrete-token inputs and continuous-embedding inputs, yielding a sequence-level reverse KL estimator for answer reliability. Our analysis shows that under certain assumptions, the expected estimate equals the mutual information between the unresolved latent state and the emitted answer token, explaining why it captures answer-relevant uncertainty rather than arbitrary uncertainty in the latent state. When the answer is deemed insufficiently reliable, CopT performs further on-policy thinking, where a second KL estimator dynamically controls draft-answer visibility, preserving useful partial information while reducing the risk of being misled by unreliable content. Across mathematics, coding, and agentic reasoning tasks, CopT improves peak accuracy by up to 23% and reduces token usage by up to 57% at comparable or higher accuracy, without any additional training. The code is available at https://github.com/sdc17/CopT.

大模型推理链式思考自洽性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。