arXiv:2605.30898cs.AIcs.CL2026-05中稿 · ICML被引 1

统一调度与推理计算,动态优化大模型的性能与成本

UniScale: Adaptive Unified Inference Scaling via Online Joint Optimization of Model Routing and Test-Time Scaling

论文配图:UniScale: Adaptive Unified Inference Scaling via Online Joint Optimization of Model Routing and Test-Time Scaling
图 1 · 摘自论文原文
  • 将模型切换与运行时计算调整整合为统一优化空间
  • 在线学习策略,在多种场景下实现更优的性价比
  • 适合需要实时响应和资源敏感的部署系统

在大型语言模型的实际部署中,如何平衡推理质量与计算成本已成为核心挑战。现有方法通常独立处理两个维度:模型路由(根据请求复杂度切换不同规模的模型)和测试时缩放(在固定模型内调整推理计算量)。然而这种分离设计存在固有局限:模型路由因可用模型规模有限,导致性能变化粗粒度且离散;单模型测试时缩放常受容量限制,随着计算增加收益递减。此外,分开处理限制了动态环境下的适应能力。为此,我们提出统一推理缩放(UIS),将模型路由与测试时缩放纳入同一优化空间。在此基础上,提出UniScale,一个在线框架,将自适应的UIS建模为上下文多臂老虎机问题,并使用LinUCB进行策略学习。该框架引入效率感知学习与成本建模,确保在高维动作空间中稳定可扩展的优化。实验表明,UniScale有效利用了UIS空间中的协同效应,在多样且动态的推理场景中持续提供更精细、更优的质量-成本权衡。

原文摘要 · Abstract (English)

In real-world deployments of large language models (LLMs), balancing inference quality and computational cost has become a central challenge. Existing approaches tackle this trade-off along two largely independent dimensions: model routing, which switches among models of different scales to match request complexity, and test-time scaling (TTS), which adjusts inference-time compute within a fixed model for fine-grained control. However, this decoupled design introduces inherent limitations. Model routing yields coarse-grained, discrete performance changes due to the sparse set of model scales, while single-model TTS often encounters capacity ceilings and exhibits diminishing returns as compute increases. Moreover, treating the two mechanisms separately restricts adaptability in dynamic inference environments. To overcome these limitations, we introduce Unified Inference Scaling (UIS), which unifies model routing and TTS in a single optimization space. Building on this formulation, we propose UniScale, an online framework that models adaptive UIS as a contextual multi-armed bandit problem and learns inference policies via LinUCB. The framework incorporates efficiency-aware learning and cost modeling to ensure stable and scalable optimization over high-dimensional action spaces. Evaluation shows that UniScale effectively exploits the synergy in the UIS space to deliver a fine-grained and consistently better quality-cost trade-off across diverse, dynamic inference scenarios.

大模型推理动态调度成本优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。