让大模型路由系统直接按精度目标自动调优，告别试错配置。
PROTEUS: SLA-Aware Routing via Lagrangian RL for Multi-LLM Serving Systems
- 用拉格朗日强化学习动态调节路由策略，实现精度目标闭环控制。
- 在多个数据集上精度达标率超90%，与理想方案差距仅1.3%~4.6%。
- 单个模型覆盖0.85~0.95全精度范围，无需重训练，适合生产部署。
生产环境中的大模型服务需应对多样负载，不同客户等级、时段和查询重要性对成本与质量要求各异。现有模型路由系统不支持直接输入延迟SLO，需离线调参并猜测准确率，参数与结果关系间接、非单调且依赖数据集。我们提出PROTEUS（面向统一SLA的多态路由器），支持运行时输入精度目标tau。其采用拉格朗日对偶控制，通过学习对偶变量lambda跟踪约束违反情况，并指导策略网络决策，使路由能精准响应指定tau。单一训练模型即可覆盖全精度范围，无需重训。在RouterBench（11模型，40.5万查询）和SPROUT（14模型，4.5万查询）上验证，PROTEUS实现稳定达标，精度满足或超过tau，目标-响应相关性达0.97~0.98。最接近的基线OmniRouter仅22%时间达标。在RouterBench上达90.1%准确率，距理想值仅差1.3%；SPROUT上达94.0%，差4.6%。相比最优固定模型，成本降低最高达89.8%。
原文摘要 · Abstract (English)
Production LLM deployments serve diverse workloads where cost and quality requirements vary by customer tier, time of day, and query criticality. Model serving systems accept latency SLOs directly. LLM routers do not. They force operators to tune parameters offline and guess what accuracy might result. The relationship between parameters and outcomes is indirect, non-monotonic, and dataset-dependent. Operators need to specify accuracy targets, not infer them from opaque settings. We present PROTEUS (Polymorphic Router for Operational Target Enforcement with Unified SLA), a router that accepts accuracy targets tau as runtime input. PROTEUS uses Lagrangian dual control. A learned dual variable lambda tracks constraint violations during training and conditions the policy network. This lets the router translate specified tau values into routing decisions that satisfy them. A single trained model serves the full accuracy spectrum without retraining.We evaluate on RouterBench (11 models, 405K queries) and SPROUT (14 models, 45K queries). PROTEUS achieves consistent floor compliance where accuracy meets or exceeds tau. The target-response correlation reaches 0.97 to 0.98. The closest baseline, OmniRouter, meets floors only 22% of the time despite also using Lagrangian optimization. PROTEUS operates across tau in [0.85, 0.95] from a single model. On RouterBench it achieves 90.1% accuracy, within 1.3% of oracle. On SPROUT it achieves 94.0% accuracy, within 4.6% of oracle. Cost savings reach 89.8% versus the best fixed model.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。