arXiv:2607.20481cs.AI2026-07

不训练模型也能智能分配计算任务,靠的是本地模型自己回答的一致性。

Routing Without Training: Controllable-Ratio LLM Offloading via Reliability Gating

论文配图:Routing Without Training: Controllable-Ratio LLM Offloading via Reliability Gating
图 1 · 摘自论文原文
  • 利用本地模型多次采样结果的一致性判断是否该本地运行或上云。
  • 在多种任务和模型上表现优于其他免训练方法,部分超越有监督训练的路由器。
  • 无需训练即可灵活调整本地与云端协作比例,适合资源受限场景。

在资源受限环境下,本地-云端协同是部署大语言模型的有效方式,但现有方法常依赖训练好的路由器或适配协同的微调,导致路由行为绑定特定运行环境。本文提出CARGO,一种免训练的路由框架:通过提示扰动采样估计本地模型推理结果的一致性,结合贝叶斯提前停止实现高效的不确定性控制,并通过轻量级部署时校准支持任意目标协作比例。在多种推理与问答任务、多类本地模型及不同规模下,CARGO consistently 超过其他免训练基线,在多个场景中甚至超越有监督学习的路由器。结果表明,有效且可适应的本地-云端协同可直接源于本地模型自身的内在响应行为,无需额外训练路由器。

原文摘要 · Abstract (English)

Local-cloud collaboration is a practical way to deploy large language models under resource constraints, but existing methods often rely on trained routers or collaboration-aware finetuning that tie routing behavior to a particular operating regime. In this work, we show that such training may be unnecessary: the local model's own inference-time agreement across sampled responses already provides a strong signal for deciding when to trust local execution and when to offload to a stronger cloud model. We propose CARGO, a training-free routing framework that estimates this agreement through prompt-varied sampling, applies Bayesian early stopping for sample-efficient uncertainty control, and supports arbitrary target collaboration ratios through lightweight deployment-time calibration. Across diverse reasoning and question-answering tasks, multiple local LLM families and scales, and both pretrained and finetuned local models, CARGO consistently outperforms other training-free baselines and in several settings surpasses supervised learned routers. These results suggest that effective and adaptable local-cloud collaboration can emerge directly from the local model's intrinsic response behavior, without requiring an additional trained router.

LLM调度免训练本地云协同

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。