arXiv:2510.09354cs.CL2025-10ACL被引 6

不训练大模型,用小模型引导实现长链条推理。

Logit Arithmetic Elicits Long Reasoning Capabilities Without Training

  • 通过日志运算在解码时迁移小模型的推理能力到大模型。
  • 相比原大模型,性能提升21.5%至24.2%。
  • 无需训练大模型,适合大规模部署的推理增强。

大型推理模型通常需要资源密集型的后训练才能实现复杂策略如回溯和自验证的长链思维。我们探究是否可在无需梯度更新的情况下激发此类行为。为此,提出一种解码阶段方法ThinkLogit,利用日志运算将一个较小的推理引导模型的能力迁移到一个大型非推理目标模型上。进一步通过偏好优化训练引导模型修正目标模型的错误,形成ThinkLogit-DPO。在数学、科学和编码领域的六个推理基准上评估,使用Qwen2.5-32B作为目标模型,由比其小21倍的R1-Distill-Qwen-1.5B引导。实验表明,ThinkLogit和ThinkLogit-DPO分别相对目标模型提升21.5%和24.2%。即使引导模型与目标模型来自不同模型家族,ThinkLogit依然有效。关键在于大模型零训练,且日志并行计算时推理开销极小,为大规模启用长推理提供了实用方案。

原文摘要 · Abstract (English)

Large reasoning models exhibit long chain-of-thought reasoning with complex strategies such as backtracking and self-verification. Yet, these capabilities typically require resource-intensive post-training. We investigate whether such behaviors can be elicited in large models without any gradient updates. To this end, we propose a decoding-time approach, ThinkLogit, which utilizes logit arithmetic to transfer these capabilities from a substantially smaller reasoning guider to a large non-reasoning target. We further show that we can boost performance by training the guider to correct the target's errors using preference optimization over mixed model outputs, a setup we refer to as ThinkLogit-DPO. We evaluate these methods across six reasoning benchmarks spanning math, science, and coding domains using the Qwen2.5-32B guided by R1-Distill-Qwen-1.5B, a model 21x smaller. Our experiments demonstrate that ThinkLogit and ThinkLogit-DPO achieve a relative improvement of 21.5% and 24.2%, respectively, over the target model. Moreover, ThinkLogit remains effective even when the guider and target come from different model families. Crucially, our method requires zero training for the large model and would incur minimal inference overhead when logits are computed in parallel, presenting a practical solution for enabling long reasoning at scale.

推理增强无训练日志运算

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。