arXiv:2605.24799cs.CVcs.AI2026-05

解决多模态大模型长序列分类性能下降问题,提升大规模图像识别精度。

Divide-and-Conquer Inference for Large-Scale Visual Recognition with Multimodal Large Language Models

论文配图:Divide-and-Conquer Inference for Large-Scale Visual Recognition with Multimodal Large Language Models
图 1 · 摘自论文原文
  • 将复杂分类任务拆解为局部子问题,动态压缩搜索空间。
  • 在ImageNet-1K和ImageNet-21K上显著提升准确率,无需微调。
  • 适用于轻量开源模型,可媲美闭源大模型,通用性强。

多模态大语言模型(MLLMs)在众多视觉语言任务中表现强劲,但在大规模图像分类任务中,随着标签空间扩大,性能显著下降,我们称之为长序列识别中的性能坍缩。通过信息论分析发现,这一现象源于信息熵上升与注意力机制中注意力稀释衰减之间的根本矛盾,导致模型在处理超长提示时信噪比不足。为此,我们提出分而治之推理(DCI),一种针对MLLM视觉识别的新型测试时扩展策略。DCI递归地将复杂的全局分类任务分解为多个更简单的局部子问题,并采用动态剪枝机制压缩搜索空间。该方法有效提升了局部信噪比,缓解了长序列推理中的权重稀释问题。此外,传统自注意力具有二次计算复杂度,而DCI实现更优的缩放行为,在大规模分类场景中显著加速推理。大量实验在ImageNet-1K和ImageNet-21K等基准上表明,DCI持续提升分类准确率,使轻量开源模型无需额外训练即可媲美甚至超越前沿闭源大模型。作为一种模型无关、即插即用的范式,DCI为提升MLLM在大规模场景下的推理精度提供了高效路径。

原文摘要 · Abstract (English)

Multimodal Large Language Models (MLLMs) have demonstrated strong capabilities across a wide range of vision language tasks. However, when applied to large scale image classification, their performance degrades significantly as the label space expands a phenomenon we define as Performance Collapse in Long Sequence Recognition. Through an information theoretic analysis, we reveal that this collapse stems from a fundamental conflict between the escalating information entropy and the prominent attention dilution and decay within attention mechanisms, which impairs the model's ability to maintain a sufficient signal-to-noise ratio when processing extremely long prompts. To mitigate this, we propose Divide-and-Conquer Inference (DCI), a novel test-time scaling strategy for visual recognition with MLLMs. DCI recursively decomposes complex global classification tasks into multiple simpler, localized subproblems and employs a dynamic pruning mechanism to compress the search space. This method effectively improves the local signal to noise ratio and model accuracy by mitigating the inherent weight dilution issues in long-sequence inference. Moreover, while traditional self-attention incurs a prohibitive quadratic computational complexity, DCI achieves more favorable scaling behavior and substantially accelerates inference in large scale classification scenarios. Extensive experiments on benchmarks such as ImageNet-1K and ImageNet-21K demonstrate that DCI consistently improves classification accuracy. This enables lightweight open-source models to rival or even surpass frontier closed-source giants without any additional training or fine-tuning. As a model-agnostic, plug-and-play paradigm, DCI offers an efficient approach for scaling the inferential precision of MLLMs in large-scale scenarios.

多模态图像分类推理优化注意力机制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。