arXiv:2607.20327cs.CL2026-07

小模型自主决定何时求助大模型,省钱又高效。

PyroDash: Cost-Efficient Token-Level Small-Large Language Model Collaborative Inference

论文配图:PyroDash: Cost-Efficient Token-Level Small-Large Language Model Collaborative Inference
图 1 · 摘自论文原文
  • 小模型生成控制符触发大模型协助,单次交接完成推理
  • 在数学推理任务上,成本降20.4%仍比纯大模型高6.36个百分点准确率
  • 适合需要平衡推理成本与性能的部署场景

大型语言模型(LLMs)具备强大推理能力但服务成本高昂,小型语言模型(SLMs)成本低却在难题上表现不佳。本文提出PyroDash,一种面向令牌级小-大模型协作推理的成本感知框架。生成过程中,小模型通过输出控制符决定是否请求帮助;协作引擎将查询和部分推理过程单次传递给冻结的大模型以完成任务。该策略内置于小模型中,无需额外路由、大模型重训练或访问其对数概率。PyroDash分三阶段训练小模型:控制符嵌入学习、面向卸载的监督微调,以及基于组相对策略优化的成本感知对齐。奖励函数权衡答案准确率与归一化后的推理成本(以纯大模型推理为基准)。在五个数学推理基准上,PyroDash支持多种精度-成本权衡点。当λ=0.05时,平均准确率达64.04%,高于大模型基线6.36个百分点,成本降低20.4%;当λ=0.6时,准确率为54.55%,大模型令牌占比仅1.90%,每例仅0.012次大模型调用,总成本从49.36美元降至1.78美元。结果表明,学习的令牌级协作可显著减少大模型使用,同时保持强推理性能。

原文摘要 · Abstract (English)

Large language models (LLMs) provide strong reasoning capabilities but are expensive to serve at scale, whereas small language models (SLMs) are cheaper but less reliable on difficult problems. We introduce PyroDash, a cost-aware framework for token-level SLM-LLM collaborative inference. During generation, the SLM decides whether to request assistance by emitting a control token. A Collaborate Engine then sends the query and partial reasoning trace to a frozen LLM for completion through a single handoff. The policy is internalized in the SLM, requiring neither a separate router, LLM retraining, nor access to LLM logits. PyroDash trains the SLM in three stages: control-token embedding learning, offloading-oriented supervised fine-tuning, and cost-aware alignment with Group Relative Policy Optimization. Its reward balances answer accuracy against inference cost normalized by LLM-only inference. Across five mathematical reasoning benchmarks, PyroDash supports different accuracy-cost operating points. With $λ=0.05$, it achieves 64.04 percent average accuracy, 6.36 percentage points above the LLM-only baseline, while reducing cost by 20.4 percent. With $λ=0.6$, it achieves 54.55 percent accuracy with a 1.90 percent LLM token ratio and 0.012 LLM calls per example, reducing total cost from USD 49.36 to USD 1.78. These results show that learned token-level handoffs can reduce LLM use while preserving strong reasoning performance.

推理优化成本控制协同推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。