让AI看图推理更深入且能自我纠错,提升复杂视觉任务准确率。
Deep But Reliable: Advancing Multi-turn Reasoning for Thinking with Images
- 通过多轮工具调用构建深度推理链,引导模型逐步分析图像。
- 在多个视觉理解基准上表现优于现有方法,错误率显著降低。
- 适合需要高可靠性推理的场景,如医疗影像分析、自动驾驶决策。
近期大型视觉语言模型(VLMs)在复杂视觉任务中展现出强大的推理能力,通过在思维链(CoT)中主动调用工具分析视觉输入,实现“看图思考”。然而,现有模型在错误推理路径上缺乏自我反思与修正能力。为此,本文提出DRIM模型,支持在多轮视觉思维链中实现深度且可靠的推理。该方法包含三个阶段:数据构建、冷启动监督微调(SFT)和强化学习(RL)。基于高分辨率图像数据集,构建高难度、可验证的视觉问答对,每个任务需多轮工具调用才能正确解答。在SFT阶段,收集工具调用轨迹作为冷启动数据,引导多轮推理模式。在RL阶段,引入冗余惩罚策略优化,鼓励模型发展自省式推理机制,即对推理路径进行判断,并惩罚未充分多尺度探索却得出错误答案的路径。大量实验表明,DRIM在多个视觉理解基准上取得优异性能。
原文摘要 · Abstract (English)
Recent advances in large Vision-Language Models (VLMs) have exhibited strong reasoning capabilities on complex visual tasks by thinking with images in their Chain-of-Thought (CoT), which is achieved by actively invoking tools to analyze visual inputs rather than merely perceiving them. However, existing models often struggle to reflect on and correct themselves when attempting incorrect reasoning trajectories. To address this limitation, we propose DRIM, a model that enables deep but reliable multi-turn reasoning when thinking with images in its multimodal CoT. Our pipeline comprises three stages: data construction, cold-start SFT and RL. Based on a high-resolution image dataset, we construct high-difficulty and verifiable visual question-answer pairs, where solving each task requires multi-turn tool calls to reach the correct answer. In the SFT stage, we collect tool trajectories as cold-start data, guiding a multi-turn reasoning pattern. In the RL stage, we introduce redundancy-penalized policy optimization, which incentivizes the model to develop a self-reflective reasoning pattern. The basic idea is to impose judgment on reasoning trajectories and penalize those that produce incorrect answers without sufficient multi-scale exploration. Extensive experiments demonstrate that DRIM achieves superior performance on visual understanding benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。