arXiv:2506.22434cs.CV2025-06NeurIPS被引 8

无需人工标注,通过对比多图实现视觉推理

MiCo: Multi-image Contrast for Reinforcement Visual Reasoning

  • 构建图像三元组,利用自监督信号训练模型比较图像异同
  • 仅用视觉对比任务训练,却在多图推理上超越基准
  • 适合需要少标注或零标注视觉逻辑推理的场景

本文探索如何让思维链(CoT)推理跨越多幅图像关联视觉线索。传统方法依赖人工标注的问题-答案对,但在细粒度视觉细节和跨图复杂逻辑下效果受限。受自监督视觉表示学习启发,我们构造包含同一图像的两个增强视图与一个相似但不同的第三张图像的三元组。训练时,模型被要求生成推理过程以判断三张图是否相同。通过基于规则的强化学习优化模型。由于图像间高度相似且存在增强,模型必须关注细微差异并进行逻辑推理才能成功。实验表明,尽管仅在视觉对比任务上训练,其推理能力可有效泛化至多种问题。无需任何人工标注的问答对,该方法在多图推理基准上取得显著提升,并在通用视觉任务中表现优异。

原文摘要 · Abstract (English)

This work explores enabling Chain-of-Thought (CoT) reasoning to link visual cues across multiple images. A straightforward solution is to adapt rule-based reinforcement learning for Vision-Language Models (VLMs). However, such methods typically rely on manually curated question-answer pairs, which can be particularly challenging when dealing with fine grained visual details and complex logic across images. Inspired by self-supervised visual representation learning, we observe that images contain inherent constraints that can serve as supervision. Based on this insight, we construct image triplets comprising two augmented views of the same image and a third, similar but distinct image. During training, the model is prompted to generate a reasoning process to compare these images (i.e., determine same or different). Then we optimize the model with rule-based reinforcement learning. Due to the high visual similarity and the presence of augmentations, the model must attend to subtle visual changes and perform logical reasoning to succeed. Experiments show that, although trained solely on visual comparison tasks, the learned reasoning ability generalizes effectively to a wide range of questions. Without relying on any human-annotated question-answer pairs, our method achieves significant improvements on multi-image reasoning benchmarks and shows strong performance on general vision tasks.

视觉推理自监督多图理解

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。