通过视觉锚定提升多模态推理准确率
SATORI-R1: Incentivizing Multimodal Reasoning through Explicit Visual Anchoring
- 分三阶段生成可验证的视觉引导推理链
- 在7个基准上最高提升15.7%准确率
- 适合需要可解释多模态推理的研究者
DeepSeek-R1 在文本领域通过稳定的强化学习展现了强大的推理能力。近期,研究尝试将强化学习直接应用于多模态任务,生成类似R1的自由形式推理以解决视觉问答(VQA)问题。然而,多模态任务与纯文本任务本质不同,高度依赖对输入图像的理解。因此,这种自由形式推理在VQA中面临两大瓶颈:(1) 推理链过长会分散对关键区域的视觉注意力,降低答案准确率;(2) 不可验证的中间步骤放大策略梯度方差并增加计算开销。为此,本文提出SATORI(空间锚定任务优化的强化学习),将VQA分解为三个可验证阶段:全局图像描述、区域定位和答案预测,每个阶段均提供显式奖励信号。同时,我们构建了VQA-Verify数据集,包含12,000条与答案对齐的描述和边界框标注,以支持训练。实验表明,在七个VQA基准上均实现一致性能提升,相比R1类基线最高提升15.7%准确率。注意力图分析证实,模型对关键区域的关注显著增强,从而带来准确率提升。代码已开源。
原文摘要 · Abstract (English)
DeepSeek-R1 has demonstrated powerful reasoning capabilities in the text domain through stable reinforcement learning (RL). Recently, in the multimodal domain, works have begun to directly apply RL to generate R1-like free-form reasoning for Visual Question Answering (VQA) tasks. However, multimodal tasks share an intrinsically different nature from textual tasks, which heavily rely on the understanding of the input image to solve the problem. Therefore, such free-form reasoning faces two critical limitations in the VQA task: (1) Extended reasoning chains diffuse visual focus away from task-critical regions, degrading answer accuracy. (2) Unverifiable intermediate steps amplify policy-gradient variance and computational costs overhead. To address these issues, in this paper, we introduce SATORI ($\textbf{S}patially$ $\textbf{A}nchored$ $\textbf{T}ask$ $\textbf{O}ptimization$ with $\textbf{R}e\textbf{I}nforcement$ Learning), which decomposes VQA into three verifiable stages, including global image captioning, region localization, and answer prediction, each supplying explicit reward signals. Furthermore, we also introduce VQA-Verify, a 12k dataset annotated with answer-aligned captions and bounding-boxes to facilitate training. Experiments demonstrate consistent performance improvements across seven VQA benchmarks, achieving up to $15.7\%$ improvement in accuracy in accuracy compared to the R1-like baseline. Our analysis of the attention map confirms enhanced focus on critical regions, which brings improvements in accuracy. Our code is available at https://github.com/justairr/SATORI-R1.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。