用大模型动态选最佳图像分类器，跨数据集更稳更可解释。
MLLM-Routed Heterogeneous Ensembles for Robust Cross-Dataset Image Classification
- 用多模态大模型做路由，自动匹配图像到最适合的视觉主干网络
- 在多个异构数据集上表现优于固定模型，准确率提升显著
- 只需改提示词就能接入新数据，适合需要灵活部署的AI系统
当前图像分类模型在单一任务数据集上表现优异，但在跨域和不同难度级别间泛化能力不足。本文提出ARMDIL，一种基于多模态大语言模型（MLLM）的自适应路由机制，用于多领域图像分类。该方法构建一个集成系统，利用MLLM代理动态将每张图像路由至最合适的视觉主干网络。集成中包含卷积神经网络（ResNets）、自监督表征学习器（SSL）和视觉语言模型（VLMs），均在统一标签空间下训练，该空间由多个分布与特性各异的图像数据集构建而成。实证评估揭示了各类架构在不同视觉域中的独特能力与脆弱性。关键在于，ARMDIL有效权衡这些差异，在跨数据集任务中表现媲美专门训练的路由模型。此外，它通过简单修改提示词即可快速融入新信息，显著提升可扩展性；并通过自然语言推理轨迹增强可解释性。这些进展为更可靠的通用视觉系统（如AI助手、自动驾驶机器人）铺平道路。
原文摘要 · Abstract (English)
Modern image classification models excel when trained on single task-specific datasets but often struggle to generalize across domains and difficulty levels. We propose ARMDIL, an Adaptive Router for Multi-Domain Image Classification with LLMs. ARMDIL is an ensemble that uses a multimodal large language model (MLLM) agent to dynamically route each image to the most suitable vision backbone. Our diverse ensemble employs convolutional neural networks (ResNets), self-supervised representation learners (SSL), and vision language models (VLMs), each trained on a unified label space constructed from multiple image datasets with differing distributions and characteristics. Empirical evaluations illuminate the distinct capabilities and vulnerabilities of each architecture across disparate visual domains. Crucially, we show that ARMDIL effectively navigates these tradeoffs, performing competitively with specialized training-based routers. Furthermore, it drastically improves adaptability by allowing new information to be integrated via simple prompt modifications, while enhancing interpretability through natural language reasoning traces. These advances in cross-dataset image classification pave the way for more reliable general-purpose vision systems such as AI assistants and autonomous robots.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。