arXiv:2604.00136cs.LGcs.CL2026-04被引 2

动态调整大模型服务路由,省钱又抗变化。

ParetoBandit: Budget-Paced Adaptive Routing for Non-Stationary LLM Serving

论文配图:ParetoBandit: Budget-Paced Adaptive Routing for Non-Stationary LLM Serving
图 1 · 摘自论文原文
  • 用在线优化算法控制每请求成本上限,无需预知总请求数
  • 在价格和质量突变时仍保持性能提升0.071,预算偏差仅0.4%
  • 适合需要长期稳定运行且频繁更新模型的服务系统

多模型大语言模型服务处于非平稳、高噪声环境中:服务商会随时调整定价,模型质量可能突然下降或提升,新模型也不断涌现。尽管已有十余种学习型路由方法试图在覆盖约530倍成本差异的模型组合中权衡质量与成本,但现有方案仍存在两大缺陷:缺乏在开放请求流中对美元成本上限的闭环约束;无法在部署后对价格或质量变化进行有原则的在线适应。本文提出ParetoBandit,一个开源自适应路由框架,基于成本感知的上下文多臂老虎机解决上述问题。其核心贡献为:(1) 一种无需已知总请求量即可实现每请求成本上限的在线原始-对偶预算调节机制;(2) 对充分统计量采用几何遗忘策略,使带宽受限的带机能够持续追踪质量与成本变化。热插拔模型注册表支持运行时模型更新,并以预算可控方式探索新模型。在包含三个模型的1,824个基准提示上测试,该路由系统维持预算合规性误差低于0.4%,在价格与质量波动时可带来最高0.071的性能增益,并可在约142步内完成冷启动模型的集成。

原文摘要 · Abstract (English)

Multi-model LLM serving operates in a non-stationary, noisy environment: providers revise pricing, model quality can shift or regress without notice, and new models arrive regularly. More than a dozen recent methods have proposed learned routers to navigate the resulting quality--cost tradeoff across portfolios spanning a $\sim$530$\times$ cost range. Despite this activity, two gaps in the current solution space limit routing effectiveness under these conditions: no existing router enforces a dollar-denominated cost ceiling in closed loop over an open-ended request stream, and none provides principled online adaptation to post-deployment shifts in pricing or model quality. We present ParetoBandit, an open-source adaptive router built on cost-aware contextual bandits that addresses both gaps. Its core contributions are: (1) an online primal--dual budget pacer that enforces a per-request cost ceiling without a known horizon, and (2) geometric forgetting on sufficient statistics that gives the bandit bounded memory for tracking quality and cost shifts. A hot-swap model registry further supports runtime model changes with budget-controlled exploration. On 1,824 benchmark prompts with a three-model portfolio, the router maintains budget compliance within 0.4%, adapts to price and quality shifts with up to +0.071 quality lift, and integrates a cold-started model within $\sim$142 steps.

大模型服务自适应路由预算控制在线学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。