用提示特征智能选大模型，兼顾效果与效率。
LLMRank: Understanding LLM Strengths for Model Routing
- 基于提示的多维度特征（任务类型、复杂度等）做路由决策。
- 在11个基准上达90%以上最优模型表现，接近理想选择。
- 结果可解释，适合需要透明部署的场景。
大型语言模型能力各异，但部署时需权衡性能与效率。我们提出LLMRank，一种提示感知的路由框架，利用从提示中提取的丰富人类可读特征——包括任务类型、推理模式、复杂度指标、句法线索及轻量代理求解器信号——而非仅依赖隐向量。该框架通过神经排序模型，在包含36,497个提示、覆盖11个基准和11个先进LLM（从小型高效模型到大型前沿系统）的RouterBench数据集上训练。实验表明，其可实现高达89.2%的理论最优利用率，并提供可解释的特征归因，揭示路由逻辑。大量研究验证了多维度特征提取与混合排序目标的重要性，展示了特征驱动路由在高效、透明部署中的潜力。
原文摘要 · Abstract (English)
The rapid growth of large language models (LLMs) with diverse capabilities, latency and computational costs presents a critical deployment challenge: selecting the most suitable model for each prompt to optimize the trade-off between performance and efficiency. We introduce LLMRank, a prompt-aware routing framework that leverages rich, human-readable features extracted from prompts, including task type, reasoning patterns, complexity indicators, syntactic cues, and signals from a lightweight proxy solver. Unlike prior one-shot routers that rely solely on latent embeddings, LLMRank predicts per-model utility using a neural ranking model trained on RouterBench, comprising 36,497 prompts spanning 11 benchmarks and 11 state-of-the-art LLMs, from small efficient models to large frontier systems. Our approach achieves up to 89.2% of oracle utility, while providing interpretable feature attributions that explain routing decisions. Extensive studies demonstrate the importance of multifaceted feature extraction and the hybrid ranking objective, highlighting the potential of feature-driven routing for efficient and transparent LLM deployment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。