让小模型主动喊大模型帮忙，只在关键步骤调用，大幅降低推理成本。
RelayLLM: Efficient Reasoning via Collaborative Decoding
- 小模型作为控制器，按需调用大模型生成关键词
- 仅调用1.07%的token，准确率达49.52%
- 适合资源受限场景下的高效复杂推理
大型语言模型（LLMs）在复杂推理中常因高计算成本和延迟受阻，而资源高效的中小型语言模型（SLMs）通常缺乏足够的推理能力。现有协作方法如级联或路由以粗粒度方式将完整查询转交给大模型，导致当小模型可处理多数推理步骤时仍产生大量计算浪费。为此，我们提出RelayLLM，一种基于令牌级协作解码的高效推理框架。与路由器不同，RelayLLM使小模型成为主动控制者，通过特殊指令仅在关键时刻动态调用大模型，实现生成过程的“接力”。我们设计了两阶段训练框架，包括预热和组相对策略优化（GRPO），以教会模型在自主性和策略性求助间取得平衡。在六个基准测试上的实证结果表明，RelayLLM实现了平均49.52%的准确率，有效弥合了两类模型间的性能差距。值得注意的是，该效果仅需调用大模型生成总令牌数的1.07%，相比性能相当的随机路由器降低了98.2%的成本。
原文摘要 · Abstract (English)
Large Language Models (LLMs) for complex reasoning is often hindered by high computational costs and latency, while resource-efficient Small Language Models (SLMs) typically lack the necessary reasoning capacity. Existing collaborative approaches, such as cascading or routing, operate at a coarse granularity by offloading entire queries to LLMs, resulting in significant computational waste when the SLM is capable of handling the majority of reasoning steps. To address this, we propose RelayLLM, a novel framework for efficient reasoning via token-level collaborative decoding. Unlike routers, RelayLLM empowers the SLM to act as an active controller that dynamically invokes the LLM only for critical tokens via a special command, effectively "relaying" the generation process. We introduce a two-stage training framework, including warm-up and Group Relative Policy Optimization (GRPO) to teach the model to balance independence with strategic help-seeking. Empirical results across six benchmarks demonstrate that RelayLLM achieves an average accuracy of 49.52%, effectively bridging the performance gap between the two models. Notably, this is achieved by invoking the LLM for only 1.07% of the total generated tokens, offering a 98.2% cost reduction compared to performance-matched random routers.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。