用小模型辅助大模型，让推理更快更准。
Accelerating Large Language Model Reasoning via Speculative Search
- 小模型预生成高质量推理路径，大模型只验证关键步骤
- 新拒收机制确保淘汰低质推理，保留与大模型相当的准确率
- 在Qwen和Llama上提速最高达2.12倍，适合高并发推理场景
基于树搜索的推理方法通过探索多个中间推理步骤显著提升了大语言模型（LLM）的推理能力，但其生成大量推理思考导致显著的推理延迟，严重限制了实际应用。为此，我们提出一种新型的推测搜索（Speculative Search, SpecSearch）框架，通过优化推理思考的生成过程大幅加速LLM推理。具体而言，SpecSearch利用小模型在思考和词元两个层面与大模型协同工作，高效生成高质量的推理路径。其核心是创新的质量保持拒绝机制，能有效过滤掉质量低于大模型输出的推理路径。实验表明，SpecSearch在保持与大模型相近推理质量的前提下，显著优于现有先进方法，在Qwen和Llama模型上实现了最高达2.12×的加速效果。
原文摘要 · Abstract (English)
Tree-search-based reasoning methods have significantly enhanced the reasoning capability of large language models (LLMs) by facilitating the exploration of multiple intermediate reasoning steps, i.e., thoughts. However, these methods suffer from substantial inference latency, as they have to generate numerous reasoning thoughts, severely limiting LLM applicability. To address this challenge, we propose a novel Speculative Search (SpecSearch) framework that significantly accelerates LLM reasoning by optimizing thought generation. Specifically, SpecSearch utilizes a small model to strategically collaborate with a large model at both thought and token levels, efficiently generating high-quality reasoning thoughts. The major pillar of SpecSearch is a novel quality-preserving rejection mechanism, which effectively filters out thoughts whose quality falls below that of the large model's outputs. Moreover, we show that SpecSearch preserves comparable reasoning quality to the large model. Experiments on both the Qwen and Llama models demonstrate that SpecSearch significantly outperforms state-of-the-art approaches, achieving up to 2.12$\times$ speedup with comparable reasoning quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。