用神经版UCB动态调度大模型，省成本还高效。
Reward-Based Online LLM Routing via NeuralUCB
- 基于神经网络的UCB算法实现动态路由决策
- 在RouterBench上比随机和最低成本基线提升收益
- 适合关注推理成本优化的部署工程师
本研究探讨了使用神经版UCB进行成本感知的大语言模型路由。现有路由方法可分为监督式和部分反馈式，各有效率与适应性的权衡。我们在模拟在线环境下，于RouterBench上实现了基于神经UCB的路由策略。实验结果表明，该方法在效用奖励上始终优于随机和最小成本基线；相较于最大质量参考模型，本方法显著降低了推理成本，同时保持了具有竞争力的奖励表现。这些发现表明，神经UCB是成本感知大模型路由的一种有前景的方法，同时也揭示了动作区分与探索方面的现存挑战。
原文摘要 · Abstract (English)
This study investigates the use of NeuralUCB for cost-aware large language model (LLM) routing. Existing routing approaches can be broadly grouped into supervised routing methods and partial-feedback methods, each with different tradeoffs in efficiency and adaptivity. We implement a NeuralUCB-based routing policy and evaluate it on RouterBench under a simulated online setting. Experimental results show that the proposed method consistently outperforms random and min-cost baselines in utility reward. Compared with the max-quality reference, our method achieves substantially lower inference cost while maintaining competitive reward. These findings suggest that NeuralUCB is a promising approach for cost-aware LLM routing, while also highlighting remaining challenges in action discrimination and exploration.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。