arXiv:2504.07891cs.LGcs.AI2025-04NeurIPS被引 64

用轻量模型预判推理步骤,加速大模型思考过程。

SpecReason: Fast and Accurate Inference-Time Compute via Speculative Reasoning

论文配图:SpecReason: Fast and Accurate Inference-Time Compute via Speculative Reasoning
图 1 · 摘自论文原文
  • 用轻量模型预判中间推理步骤,仅在必要时调用主模型验证
  • 在多个基准上实现1.4-3.0倍加速,准确率提升0.4-9.0%
  • 适合追求高效率推理的开发者和部署场景

近期推理阶段计算的进步通过使用大型推理模型(LRMs)生成长链条思维(CoTs)显著提升了复杂任务的表现。然而,这种精度提升带来了高延迟,源于生成序列过长及解码的自回归特性。我们观察到LRM推理及其内嵌的推理过程对近似具有高度容忍性:复杂任务通常被分解为简单步骤,每个步骤的价值在于其语义洞察对后续步骤的帮助,而非生成的具体标记。为此,我们提出SpecReason,一个通过轻量模型(推测性地)执行简单中间推理步骤,并仅在必要时调用昂贵的基础模型来评估(并可能修正)推测输出,从而自动加速LRM推理的系统。SpecReason关注于利用思维标记的语义灵活性以保持最终答案准确性,这与以往推测技术(如推测解码)互补,后者要求每一步的标记级等价性。在多种推理基准上,SpecReason相比原始LRM推理实现1.4–3.0倍加速,同时准确率提升0.4–9.0%。与推测解码结合后,额外降低8.8–58.0%延迟。项目已开源:https://github.com/ruipeterpan/specreason。

原文摘要 · Abstract (English)

Recent advances in inference-time compute have significantly improved performance on complex tasks by generating long chains of thought (CoTs) using Large Reasoning Models (LRMs). However, this improved accuracy comes at the cost of high inference latency due to the length of generated reasoning sequences and the autoregressive nature of decoding. Our key insight in tackling these overheads is that LRM inference, and the reasoning that it embeds, is highly tolerant of approximations: complex tasks are typically broken down into simpler steps, each of which brings utility based on the semantic insight it provides for downstream steps rather than the exact tokens it generates. Accordingly, we introduce SpecReason, a system that automatically accelerates LRM inference by using a lightweight model to (speculatively) carry out simpler intermediate reasoning steps and reserving the costly base model only to assess (and potentially correct) the speculated outputs. Importantly, SpecReason's focus on exploiting the semantic flexibility of thinking tokens in preserving final-answer accuracy is complementary to prior speculation techniques, most notably speculative decoding, which demands token-level equivalence at each step. Across a variety of reasoning benchmarks, SpecReason achieves $1.4-3.0\times$ speedup over vanilla LRM inference while improving accuracy by $0.4-9.0\%$. Compared to speculative decoding without SpecReason, their combination yields an additional $8.8-58.0\%$ latency reduction. We open-source SpecReason at https://github.com/ruipeterpan/specreason.

推理加速推测推理大模型优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。