arXiv:2601.05110cs.AI2026-01ACL被引 12

用首个思维标记的熵值判断推理难度,实现高效协同推理。

GlimpRouter: Efficient Collaborative Inference by Glimpsing One Token of Thoughts

  • 仅看每步第一个词的不确定性,决定是否调用大模型。
  • 在AIME25上降低25.9%延迟,准确率提升10.7%。
  • 无需训练,适合部署在资源受限的推理场景。

大型推理模型(LRMs)通过显式生成多步思维链取得优异表现,但带来显著的推理延迟和计算开销。协同推理通过在轻量与大型模型间选择性分配任务提供了解决方案,但核心挑战在于:如何判断某一步推理需大模型能力还是小模型效率。现有路由策略依赖局部词概率或事后验证,引入额外延迟。本文提出新视角:推理步骤的难度可从其首个词中推断。受“顿悟时刻”现象启发,我们发现初始词的熵是步骤难度的强预测因子。基于此,提出GlimpRouter——一种无需训练的分步协同框架。该方法用轻量模型仅生成每步首个词,当初始词熵超过阈值时,才将该步路由至大模型。多基准测试表明,该方法显著降低推理延迟并保持精度。例如,在AIME25上,相比独立大模型,准确率提升10.7%,延迟降低25.9%。结果表明:基于思维初现的计算分配,是一种简单而有效的推理机制。

原文摘要 · Abstract (English)

Large Reasoning Models (LRMs) achieve remarkable performance by explicitly generating multi-step chains of thought, but this capability incurs substantial inference latency and computational cost. Collaborative inference offers a promising solution by selectively allocating work between lightweight and large models, yet a fundamental challenge remains: determining when a reasoning step requires the capacity of a large model or the efficiency of a small model. Existing routing strategies either rely on local token probabilities or post-hoc verification, introducing significant inference overhead. In this work, we propose a novel perspective on step-wise collaboration: the difficulty of a reasoning step can be inferred from its very first token. Inspired by the "Aha Moment" phenomenon in LRMs, we show that the entropy of the initial token serves as a strong predictor of step difficulty. Building on this insight, we introduce GlimpRouter, a training-free step-wise collaboration framework. GlimpRouter employs a lightweight model to generate only the first token of each reasoning step and routes the step to a larger model only when the initial token entropy exceeds a threshold. Experiments on multiple benchmarks demonstrate that our approach significantly reduces inference latency while preserving accuracy. For instance, GlimpRouter attains a substantial 10.7% improvement in accuracy while reducing inference latency by 25.9% compared to a standalone large model on AIME25. These results suggest a simple yet effective mechanism for reasoning: allocating computation based on a glimpse of thought rather than full-step evaluation.

协同推理思维链效率优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。