智能分配大模型请求，兼顾性能与成本。
OrcaRouter: A Production-Oriented LLM Router with Hybrid Offline-Online Learning
- 结合上下文特征与在线学习，动态选择最优模型。
- 部署时准确率达75.54%，每千次请求仅需1美元。
- 适合需要高效调度多模型的生产系统使用。
大规模语言模型各具能力与推理成本，如何为请求选择最佳模型成为实际部署难题。我们提出OrcaRouter，一种面向生产的LLM路由系统，融合基于林乌尔克（LinUCB）的上下文增强强化学习机制与离线-在线混合学习策略。离线阶段，通过在精心设计的路由提示集上评估所有候选模型，获得完整反馈并构建奖励矩阵，训练每个模型分支的岭回归器；部署时，初始化参数并可继续根据带宽反馈更新选中模型的分支。在2026年5月20日提交时，OrcaRouter-Adaptive在公开的RouterArena排行榜上位列第二，得分为72.08，实现每千次请求1.00美元成本下的75.54%准确率。
原文摘要 · Abstract (English)
The rapid development of large language models, each with distinct capabilities and inference costs, raises a practical deployment question: given an incoming request, which model should handle it? We present OrcaRouter, a production-oriented LLM router that combines a LinUCB-based contextual bandit over lexical and sentence-embedding features with a hybrid offline-online learning protocol. Offline, OrcaRouter obtains full-information feedback by evaluating each candidate model on a curated set of routing prompts, yielding a reward matrix used to fit one ridge regressor per arm. At deployment time, it initializes from these parameters and can optionally continue learning from bandit feedback, updating only the selected model's arm after observing its reward. At the time of our RouterArena submission (May 20, 2026), OrcaRouter-Adaptive ranked second on the public RouterArena leaderboard with an arena score of 72.08, achieving 75.54% accuracy at a cost of USD 1.00 per 1,000 queries.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。