通过动态切换模型，让复杂推理更高效且不丢精度。
RelayGen: Intra-Generation Model Switching for Efficient Reasoning
- 根据推理过程的难易变化，自动切换大小模型
- 在多个基准上实现2.2倍加速，精度损失小于2%
- 无需训练，适合部署在资源受限场景
大型推理模型（LRMs）通过生成长序列多步推理轨迹，在复杂推理任务中表现优异，但推理时扩展带来高昂部署成本。现有方法或忽略单个输出内部的难度变化，或依赖高复杂度的监督式分词级路由。我们提出 extbf{RelayGen}，一种无需训练、基于段落级别的运行时模型切换框架，利用长推理轨迹中的难度变化。通过离线分析生成不确定性（基于词元概率间距），我们发现粗粒度段级控制足以捕捉推理过程中的难度过渡。RelayGen 识别出能指示进入低难度段的模型特定切换信号，并将后续推理交给小模型处理，同时在大模型上保留高难度部分。在多个推理基准上，RelayGen 显著降低推理延迟，同时保持大部分大模型精度。结合推测解码后，可实现最高 2.2× 的端到端加速，精度损失低于 2%，且无需额外训练或学习路由模块。
原文摘要 · Abstract (English)
Large reasoning models (LRMs) achieve strong performance on complex reasoning tasks by generating long, multi-step reasoning trajectories, but inference-time scaling incurs substantial deployment cost. A key challenge is that generation difficulty varies within a single output, whereas existing efficiency-oriented approaches either ignore this intra-generation variation or rely on supervised token-level routing with high system complexity. We present \textbf{RelayGen}, a training-free, segment-level runtime model switching framework that exploits difficulty variation in long-form reasoning. Through offline analysis of generation uncertainty using token probability margins, we show that coarse-grained segment-level control is sufficient to capture difficulty transitions within a reasoning trajectory. RelayGen identifies model-specific switch cues that signal transitions to lower-difficulty segments and dynamically delegates their continuation to a smaller model, while preserving high-difficulty reasoning on the large model. Across multiple reasoning benchmarks, RelayGen substantially reduces inference latency while preserving most of the accuracy of large models. When combined with speculative decoding, RelayGen achieves up to 2.2$\times$ end-to-end speedup with less than 2\% accuracy degradation, without requiring additional training or learned routing components.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。