提出低成本开放集视觉语言模型路由方法,提升任务执行效率。
SCOPE-Router: Cost-Aware Open-Set VLM Routing for Execution-Oriented Tasks

- 构建双塔路由结构,通过混合校准匹配查询与模型行为
- 在开放集场景下性能领先,超越次优方案6.75分
- 适合需要多模型动态选型的智能系统开发者
模型路由旨在为每个查询从候选模型池中选择最合适的模型,平衡效果与成本。现有视觉语言模型(VLM)路由研究局限于传统VQA评估,缺乏对开放集场景的系统性校准优化,且训练目标通过softmax归一化稀释了多正样本信号,未考虑成本因素。本文提出三项贡献:(1) VLM-ExecRouterBench,首个面向代码、代理和搜索领域的执行导向VLM路由基准,涵盖11个模型,价格跨度近两个数量级;(2) SCOPE-Router,一种双塔路由架构,通过随机/诊断/多样性采样构建模型行为特征,支持新模型无缝接入而无需重训;(3) CRM+RCCR,一种与架构无关的成本感知目标,通过成对独立打分将成本偏好编码为连续相关性目标,消除多正样本稀释问题,并使具有相似路由偏好的查询在路由空间中更接近。实验表明,SCOPE-Router在三个基准上均取得最佳排名得分,在分布外(OOD)设置下比次优方案高1.84分,双重分布外开放集评估下高6.75分;应用于四个不同路由器时,CRM+RCCR使排名得分提升1.25至6.21分。
原文摘要 · Abstract (English)
Model routing aims to select the most suitable model from a candidate pool for each query, balancing quality and cost. Existing VLM routing research is limited to traditional VQA evaluation, lacks systematic calibration optimization for open-set scenarios, and employs training objectives that dilute multi-positive signals via softmax normalization without incorporating cost. We address these limitations with three contributions: (1)VLM-ExecRouterBench, the first execution-oriented VLM routing benchmark covering Code, Agentic, and Search domains with 11 candidate models spanning nearly two orders of magnitude in pricing; (2)SCOPE-Router, a dual-tower router that matches queries to model behavior profiles constructed via hybrid calibration (random/diagnostic/diversity sampling), enabling new models to join routing without retraining; (3)CRM+RCCR, an architecture-agnostic cost-aware objective that encodes cost preference into continuous relevance targets through per-pair independent scoring, eliminating multi-positive dilution while regularizing queries with similar routing preferences to be closer in the routing space. Empirically, SCOPE-Router achieves the best Rank Score on all three benchmarks, surpassing the runner-up by 1.84 points under OOD settings and by 6.75 points under doubly OOD open-set evaluation. When applied to four diverse routers, CRM+RCCR improves Rank Score by 1.25--6.21 points.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。