arXiv:2601.17814cs.AI2026-01被引 10

为多模态大模型设计智能路由系统,按任务自动选最优模型以省算力提精度。

MMR-Bench: A Comprehensive Benchmark for Multimodal LLM Routing

  • 构建可控环境,用多模态输入和可调算力预算测试模型路由效果。
  • 路由系统在33%成本下超越最强单模型的准确率,提升效率与性能平衡。
  • 策略可零样本迁移至新数据集,适合研究高效多模态模型部署者。

多模态大语言模型(MLLMs)发展迅速,但架构、对齐策略和效率差异导致没有单一模型能在所有任务上表现最优。实际应用中,任务从轻量级OCR到复杂多模态推理不等,统一使用一个模型会过度消耗算力或牺牲准确率。查询级模型选择(路由)可缓解此矛盾,但将文本路由扩展至多模态场景面临模态融合、模型间计算成本差异大及缺乏标准化评估等挑战。本文提出MMR-Bench,一个统一基准,隔离多模态路由问题并支持在固定候选集和成本模型下进行比较。该基准包含:(i) 模态感知输入与可变算力预算的受控环境;(ii) 覆盖OCR、通用VQA和多模态数学推理的广泛视觉-语言任务;(iii) 强单模型基线、理想路由上限及代表性路由策略。实验表明,引入多模态信号显著提升路由质量,使系统在约33%成本下超越最强单模型的准确率。此外,基于部分模型与任务训练的策略可零样本泛化至新数据集和纯文本基准,无需微调,确立了MMR-Bench作为自适应多模态模型选择与高效部署研究的基础。代码将开源于:https://github.com/Hunter-Wrynn/MMR-Bench。

原文摘要 · Abstract (English)

Multimodal large language models (MLLMs) have advanced rapidly, yet heterogeneity in architecture, alignment strategies, and efficiency means that no single model is uniformly superior across tasks. In practical deployments, workloads span lightweight OCR to complex multimodal reasoning; using one MLLM for all queries either over-provisions compute on easy instances or sacrifices accuracy on hard ones. Query-level model selection (routing) addresses this tension, but extending routing from text-only LLMs to MLLMs is nontrivial due to modality fusion, wide variation in computational cost across models, and the absence of a standardized, budget-aware evaluation. We present MMR-Bench, a unified benchmark that isolates the multimodal routing problem and enables comparison under fixed candidate sets and cost models. MMR-Bench provides (i) a controlled environment with modality-aware inputs and variable compute budgets, (ii) a broad suite of vision-language tasks covering OCR, general VQA, and multimodal math reasoning, and (iii) strong single-model reference, oracle upper bounds, and representative routing policies. Using MMR-Bench, we show that incorporating multimodal signals improves routing quality. Empirically, these cues improve the cost-accuracy frontier and enable the routed system to exceed the strongest single model's accuracy at roughly 33% of its cost. Furthermore, policies trained on a subset of models and tasks generalize zero-shot to new datasets and text-only benchmarks without retuning, establishing MMR-Bench as a foundation for studying adaptive multimodal model selection and efficient MLLM deployment. The code will be available at: https://github.com/Hunter-Wrynn/MMR-Bench.

多模态模型路由效率优化基准测试

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。