arXiv:2512.05033cs.CLcs.AI2025-12被引 2

通过动态路由提升推理效率,让模型自动选择更优生成路径。

Arbitrage: Efficient Reasoning via Advantage-Aware Speculation

  • 基于模型优势预测动态决定是否采纳草稿步骤
  • 在数学推理任务中将延迟降低近2倍,准确率不变
  • 适合追求高效推理的部署场景,尤其适合长链思维任务

现代大语言模型虽具备强大推理能力,但因采用长链思维导致推理成本高昂。为改善性能与成本的比率,现有方法如推测解码通过快速但不准确的草稿模型生成候选词,再由更强的目标模型并行验证。然而,传统逐标记验证常因语义等价但形式不同的步骤被错误拒绝,效率受限。尽管近期转向步骤级语义验证可提高效率,但现有方法仍需重生成大量被拒步骤,浪费目标模型算力。为此,本文提出Arbitrage,一种新的步骤级推测生成框架,通过轻量级路由器动态判断目标模型是否能产生显著更优的推理步骤,实现接近理想仲裁器的决策效果,从而在效率与准确性间取得近乎最优平衡。在多个数学推理基准测试中,Arbitrage持续超越先前步骤级推测解码基线,在保持相同准确率下将推理延迟降低约2倍。

原文摘要 · Abstract (English)

Modern Large Language Models achieve impressive reasoning capabilities with long Chain of Thoughts, but they incur substantial computational cost during inference, and this motivates techniques to improve the performance-cost ratio. Among these techniques, Speculative Decoding accelerates inference by employing a fast but inaccurate draft model to autoregressively propose tokens, which are then verified in parallel by a more capable target model. However, due to unnecessary rejections caused by token mismatches in semantically equivalent steps, traditional token-level Speculative Decoding struggles in reasoning tasks. Although recent works have shifted to step-level semantic verification, which improve efficiency by accepting or rejecting entire reasoning steps, existing step-level methods still regenerate many rejected steps with little improvement, wasting valuable target compute. To address this challenge, we propose Arbitrage, a novel step-level speculative generation framework that routes generation dynamically based on the relative advantage between draft and target models. Instead of applying a fixed acceptance threshold, Arbitrage uses a lightweight router trained to predict when the target model is likely to produce a meaningfully better step. This routing approximates an ideal Arbitrage Oracle that always chooses the higher-quality step, achieving near-optimal efficiency-accuracy trade-offs. Across multiple mathematical reasoning benchmarks, Arbitrage consistently surpasses prior step-level Speculative Decoding baselines, reducing inference latency by up to $\sim2\times$ at matched accuracy.

推理加速推测解码动态路由大模型优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。