用强化学习让模型像人一样按目标选视觉重点,无需标注。
VisRL: Intention-Driven Visual Perception via Reinforced Reasoning
- 用强化学习自动决定看哪里,不再依赖人工标注框
- 在多个基准上超越基线,跨模型泛化能力强
- 适合想提升视觉推理能力的研究者和开发者
视觉理解本质上是目标驱动的——人类会根据目标选择性关注场景的不同区域。近年来的大规模多模态模型(LMMs)可通过自然语言灵活表达这种意图,使查询引导视觉推理过程。如视觉思维链框架已证明显式推理步骤的价值,即模型在回答前预测关注区域。但现有方法严重依赖带注释的中间边界框进行监督训练,因意图-区域组合爆炸而难以扩展。为此,我们提出VisRL,首个将强化学习(RL)应用于目标驱动视觉感知的框架。VisRL仅使用奖励信号优化整个视觉推理过程,将中间关注区域选择视为通过试错优化的内部决策,彻底消除对昂贵区域标注的需求,更贴近人类学习感知的方式。大量实验表明,VisRL在多个基准上持续优于强基线,展现出色的有效性和跨不同LMM的强泛化能力。代码已公开于https://github.com/zhangquanchen/VisRL。
原文摘要 · Abstract (English)
Visual understanding is inherently intention-driven - humans selectively focus on different regions of a scene based on their goals. Recent advances in large multimodal models (LMMs) enable flexible expression of such intentions through natural language, allowing queries to guide visual reasoning processes. Frameworks like Visual Chain-of-Thought have demonstrated the benefit of incorporating explicit reasoning steps, where the model predicts a focus region before answering a query. However, existing approaches rely heavily on supervised training with annotated intermediate bounding boxes, which severely limits scalability due to the combinatorial explosion of intention-region pairs. To overcome this limitation, we propose VisRL, the first framework that applies reinforcement learning (RL) to the problem of intention-driven visual perception. VisRL optimizes the entire visual reasoning process using only reward signals. By treating intermediate focus selection as an internal decision optimized through trial-and-error, our method eliminates the need for costly region annotations while aligning more closely with how humans learn to perceive the world. Extensive experiments across multiple benchmarks show that VisRL consistently outperforms strong baselines, demonstrating both its effectiveness and its strong generalization across different LMMs. Our code is available at https://github.com/zhangquanchen/VisRL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。