用单头注意力动态选最合适的大模型，省钱又提效。
One Head, Many Models: Cross-Attention Routing for Cost-Aware LLM Selection
- 用单头交叉注意力联合建模查询与模型特征，实现动态选型。
- 在RouterBench上提升6.6%质量指标，最高性能增2.9%。
- 轻量通用，适合需平衡成本与效果的部署场景。
大规模语言模型（LLMs）在计算成本和性能表现上差异显著，给实际应用中的可扩展、低成本部署带来挑战。本文提出一种统一路由框架，利用单头交叉注意力机制联合建模查询与模型嵌入，实现对每个输入查询的动态最优模型选择。该方法在公开的大规模基准RouterBench上进行评估，涵盖多样化的模型池与领域。通过显式捕捉细粒度的查询-模型交互，路由器能同时预测响应质量和生成成本，在平均质量提升（AIQ）上达到最高6.6%的改进，最大性能提升达2.9%。为稳健平衡性能与成本，提出指数奖励函数，增强在不同用户偏好下的稳定性。所提架构轻量且跨领域泛化能力强，相较已有方法效率更高，树立了新的成本感知大模型路由标准。
原文摘要 · Abstract (English)
The proliferation of large language models (LLMs) with varying computational costs and performance profiles presents a critical challenge for scalable, cost-effective deployment in real-world applications. We introduce a unified routing framework that leverages a single-head cross-attention mechanism to jointly model query and model embeddings, enabling dynamic selection of the optimal LLM for each input query. Our approach is evaluated on RouterBench, a large-scale, publicly available benchmark encompassing diverse LLM pools and domains. By explicitly capturing fine-grained query-model interactions, our router predicts both response quality and generation cost, achieving up to 6.6% improvement in Average Improvement in Quality (AIQ) and 2.9% in maximum performance over existing routers. To robustly balance performance and cost, we propose an exponential reward function that enhances stability across user preferences. The resulting architecture is lightweight, generalizes effectively across domains, and demonstrates improved efficiency compared to prior methods, establishing a new standard for cost-aware LLM routing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。