通过追踪思维线索,实现高效推理模型解码,显著降低计算量。
What makes Reasoning Models Different? Follow the Reasoning Leader for Efficient Decoding
- 用主导模型引导每句开头,弱模型续写其余内容,实现快慢协同推理。
- 在4个数学推理基准上减少30%~50%理论算力,缩短思维链40%。
- 适合追求推理效率与性能平衡的开发者,尤其适用于长链条任务。
大型推理模型(LRMs)通过生成长串思维链实现强推理能力,但冗长过程导致推理变慢且易偏离主题(过思现象)。我们系统分析了推理与非推理模型在词元层面的差异,发现其核心差异不仅在于风格化的‘思考提示’,更存在两个此前未被充分关注的现象:全局不一致反弹(随响应长度增加,差异持续或加剧),以及局部不一致消减(不一致集中于每句开头的思考提示,随后迅速下降)。基于局部不一致消减,我们提出FoReaL-Decoding,一种协作式快速-慢速思维解码方法。该方法中,主导模型负责每句前几个词元,较弱的草稿模型完成后续内容;通过随机门控实现大小模型间平滑过渡。在四个主流数学推理基准(AIME24、GPQA-Diamond、MATH500、AMC23)上,该方法减少30%~50%理论浮点运算量,思维链长度缩短最多达40%,同时保持86%~100%的模型性能。这确立了FoReaL-Decoding作为推理任务中可控成本-质量权衡的简单即插即用方案。
原文摘要 · Abstract (English)
Large reasoning models (LRMs) achieve strong reasoning performance by emitting long chains of thought. Yet, these verbose traces slow down inference and often drift into unnecessary detail, known as the overthinking phenomenon. To better understand LRMs' behavior, we systematically analyze the token-level misalignment between reasoning and non-reasoning models. While it is expected that their primary difference lies in the stylistic "thinking cues", LRMs uniquely exhibit two pivotal, previously under-explored phenomena: a Global Misalignment Rebound, where their divergence from non-reasoning models persists or even grows as response length increases, and more critically, a Local Misalignment Diminish, where the misalignment concentrates at the "thinking cues" each sentence starts with but rapidly declines in the remaining of the sentence. Motivated by the Local Misalignment Diminish, we propose FoReaL-Decoding, a collaborative fast-slow thinking decoding method for cost-quality trade-off. In FoReaL-Decoding, a Leading model leads the first few tokens for each sentence, and then a weaker draft model completes the following tokens to the end of each sentence. FoReaL-Decoding adopts a stochastic gate to smoothly interpolate between the small and the large model. On four popular math-reasoning benchmarks (AIME24, GPQA-Diamond, MATH500, AMC23), FoReaL-Decoding reduces theoretical FLOPs by 30 to 50% and trims CoT length by up to 40%, while preserving 86 to 100% of model performance. These results establish FoReaL-Decoding as a simple, plug-and-play route to controllable cost-quality trade-offs in reasoning-centric tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。