让AI学会像人一样看图思考，用强化学习实现端到端训练。
DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning
- 通过强化学习让模型主动选择关键视觉信息进行推理
- 在多个基准上提升视觉理解与数学推理能力，减少幻觉
- 无需预标注数据，适合希望提升AI视觉思维能力的研究者
大型视觉语言模型在多模态理解方面表现优异，但在将视觉信息深度融入以文本为主导的推理过程方面仍存在挑战，这阻碍了其模拟人类认知的能力。为此，我们提出DeepEyes，一个通过强化学习端到端训练的模型，能够原生地‘用图像思考’，无需依赖预收集的推理数据进行冷启动监督微调。该能力源于模型自身的视觉定位能力，不依赖外部专用模型或API。我们通过主动感知机制，使模型学习战略性地基于视觉信息进行推理，并采用定制的数据选择与奖励策略加以引导。DeepEyes在通用感知与推理基准上取得显著性能提升，并在视觉定位、幻觉抑制及数学推理任务中表现更优。有趣的是，我们观察到主动感知从初始探索逐渐演变为高效精准的利用，且涌现出多样化的思维模式，与人类视觉推理过程高度相似。代码已开源：https://github.com/Visual-Agent/DeepEyes。
原文摘要 · Abstract (English)
Large Vision-Language Models excel at multimodal understanding but struggle to deeply integrate visual information into their predominantly text-based reasoning processes, a key challenge in mirroring human cognition. To address this, we introduce DeepEyes, a model that learns to "think with images", trained end-to-end with reinforcement learning without requiring pre-collected reasoning data for cold-start supervised fine-tuning (SFT). Notably, this ability emerges natively, leveraging the model's own grounding capability as an intrinsic function rather than relying on external specialized models or APIs. We enable this capability through active perception, where the model learns to strategically ground its reasoning in visual information, guided by a tailored data selection and reward strategy. DeepEyes achieves significant performance gains on general perception and reasoning benchmarks and also demonstrates improvement in grounding, hallucination, and mathematical reasoning tasks. Interestingly, we observe the distinct evolution of active perception from initial exploration to efficient and accurate exploitation, and diverse thinking patterns that closely mirror human visual reasoning processes. Code is available at https://github.com/Visual-Agent/DeepEyes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。