arXiv:2601.07298cs.CV2026-01中稿 · CVPR

模仿人类认知流程,用五步推理框架提升多图理解能力。

Mimic Human Cognition, Master Multi-Image Reasoning: A Meta-Action Framework for Enhanced Visual Understanding

  • 将多图推理拆解为聚焦、提示、思考等五步认知动作。
  • 在56000个冷启动和58000个强化学习样本上训练,性能超越GPT-4o。
  • 适合需要复杂视觉推理的场景,如视频理解与多图问答。

虽然多模态大语言模型在单图理解上表现优异,但在多图推理任务中性能显著下降。多图推理面临图像间关系复杂、关键信息分散等挑战。受人类认知过程启发,我们提出一种认知驱动的元动作框架(CINEMA),将多图推理分解为全局、聚焦、提示、思考、回答五步结构化元动作,显式建模人类自然的推理步骤。针对冷启动训练,我们设计基于检索的树采样策略,生成高质量元动作轨迹以引导模型学习推理模式。在强化学习阶段,采用两阶段范式:先通过保多样性策略探索,避免熵崩溃;再通过DAPO算法渐进增强利用。为训练模型,我们构建了包含5.6万条冷启动与5.8万条强化学习实例的数据集,覆盖多图、多帧及单图任务。在多图推理、视频理解及单图基准上广泛评估,结果表明模型在多个关键基准上达到领先水平,在MUIR和MVMath上超越GPT-4o,且显著优于专用视频推理模型,验证了该认知启发框架的有效性与泛化能力。

原文摘要 · Abstract (English)

While Multimodal Large Language Models (MLLMs) excel at single-image understanding, they exhibit significantly degraded performance in multi-image reasoning scenarios. Multi-image reasoning presents fundamental challenges including complex inter-relationships between images and scattered critical information across image sets. Inspired by human cognitive processes, we propose a Cognition-Inspired Meta-Action Framework (CINEMA), which decomposes multi-image reasoning into five structured meta-actions: Global, Focus, Hint, Think, and Answer, explicitly modeling the sequential cognitive steps humans naturally employ. For cold-start training, we introduce a Retrieval-Based Tree Sampling strategy that generates high-quality meta-action trajectories to bootstrap the model with reasoning patterns. During reinforcement learning, we adopt a two-stage paradigm: an exploration phase with Diversity-Preserving Strategy to avoid entropy collapse, followed by an annealed exploitation phase with DAPO to gradually strengthen exploitation. To train our model, we construct a dataset of 56k cold-start and 58k reinforcement learning instances spanning multi-image, multi-frame, and single-image tasks. We conduct extensive evaluations on multi-image reasoning benchmarks, video understanding benchmarks, and single-image benchmarks, achieving competitive state-of-the-art performance on several key benchmarks. Our model surpasses GPT-4o on the MUIR and MVMath benchmarks and notably outperforms specialized video reasoning models on video understanding benchmarks, demonstrating the effectiveness and generalizability of our human cognition-inspired reasoning framework.

多图推理认知模型强化学习视觉理解

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。