用对比反馈优化大模型路由，更省成本且自适应。
LLM Routing with Dueling Feedback
- 基于成对偏好学习，不依赖绝对评分，标签效率高。
- 在两个数据集上累计遗憾更低，收敛更快，性能更稳。
- 适合需要动态调度模型的生产级推理系统使用。
我们研究大模型路由问题，即在保证用户满意度、模型专长和推理成本之间平衡的前提下，为每个查询选择最优模型。将路由建模为上下文双人对抗强化学习（contextual dueling bandits），从成对偏好反馈中学习，而非依赖绝对分数，从而实现标签高效与动态适应。在此基础上，提出类别校准微调（CCFT），一种利用离线数据通过带类别权重的对比微调生成模型嵌入的表示学习方法。这些嵌入使感觉良好汤普森采样（FGTS.CDB）得以实际应用，这是一种理论严谨的后验采样算法。我们设计了四种类别权重变体，显式融合模型质量与成本信息。在RouterBench和MixInstruct数据集上进行实证评估，结果表明，所提方法在两个基准上均实现更低的累计遗憾与更快收敛速度，且相比采用通用OpenAI嵌入模型构建的强基线，展现出更强鲁棒性与性能-成本平衡能力。
原文摘要 · Abstract (English)
We study LLM routing, the problem of selecting the best model for each query while balancing user satisfaction, model expertise, and inference cost. We formulate routing as contextual dueling bandits, learning from pairwise preference feedback rather than absolute scores, thereby yielding label-efficient and dynamic adaptation. Building on this formulation, we introduce Category-Calibrated Fine-Tuning (CCFT), a representation-learning method that derives model embeddings from offline data using contrastive fine-tuning with categorical weighting. These embeddings enable the practical instantiation of Feel-Good Thompson Sampling for Contextual Dueling Bandits (FGTS.CDB), a theoretically grounded posterior-sampling algorithm. We propose four variants of the categorical weighting that explicitly integrate model quality and cost, and we empirically evaluate the proposed methods on the RouterBench and MixInstruct datasets. Across both benchmarks, our methods achieve lower cumulative regret and faster convergence, with better robustness and performance-cost balance than strong baselines built with a general-purpose OpenAI embedding model.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。