用智能路由让不同任务走最适合的模型，省钱又高效。
Towards Resource-Efficient Multimodal Intelligence: Learned Routing among Specialized Expert Models
- 根据任务类型自动选配最优专家模型，动态调度提升效率。
- 在MMLU和VQA上性能媲美顶级大模型，但减少67%以上高成本模型调用。
- 适合追求低成本、高可用多模态AI部署的开发者与企业。
随着AI拓展至视觉、音频和文档理解等领域，大型语言模型（LLMs）虽能力强大，但推理成本高昂，难以实现实时、可扩展部署。相比之下，小型开源模型虽成本低，却难以应对复杂或跨模态任务。本文提出一种统一、模块化的框架，通过学习得到的路由网络，将文本、多模态或复杂查询智能分配给最合适的专家模型，实现成本与质量的平衡。针对视觉任务，采用两阶段开源流水线，优化效率并重新启用仍为子任务最优的经典视觉组件。在Massive Multitask Language Understanding（MMLU）和Visual Question Answering（VQA）等基准测试中，性能达到或超过始终使用高端大模型（即单一模型处理所有任务的单体系统）的表现，同时对昂贵模型的依赖降低超过67%。该框架具备可扩展的多智能体协同能力，可在大规模场景下提供高质量且资源高效的AI服务。
原文摘要 · Abstract (English)
As AI moves beyond text, large language models (LLMs) increasingly power vision, audio, and document understanding; however, their high inference costs hinder real-time, scalable deployment. Conversely, smaller open-source models offer cost advantages but struggle with complex or multimodal queries. We introduce a unified, modular framework that intelligently routes each query - textual, multimodal, or complex - to the most fitting expert model, using a learned routing network that balances cost and quality. For vision tasks, we employ a two-stage open-source pipeline optimized for efficiency and reviving efficient classical vision components where they remain SOTA for sub-tasks. On benchmarks such as Massive Multitask Language Understanding (MMLU) and Visual Question Answering (VQA), we match or exceed the performance of always-premium LLM (monolithic systems with one model serving all query types) performance, yet reduce the reliance on costly models by over 67%. With its extensible, multi-agent orchestration, we deliver high-quality, resource-efficient AI at scale.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。