arXiv:2505.15804cs.CV2025-05被引 22

让AI像人一样看图推理物体空间变化,性能提升23%。

STAR-R1: Spatial TrAnsformation Reasoning by Reinforcing Multimodal LLMs

  • 用强化学习+精细奖励机制,引导模型逐步推理物体变换
  • 跨视角任务准确率比传统方法高23%,11项指标全领先
  • 能主动比较所有物体,模拟人类的空间思考方式

多模态大语言模型在各类任务中表现优异,但在空间推理上仍远落后于人类。我们通过变换驱动视觉推理(TVR)研究这一差距,该任务要求在不同视角图像间识别物体的变换。传统监督微调(SFT)在跨视角场景下无法生成连贯推理路径,而稀疏奖励强化学习存在探索效率低、收敛慢的问题。为此,我们提出STAR-R1框架,采用单阶段强化学习与针对TVR设计的细粒度奖励机制,奖励部分正确推理,惩罚过度列举和被动不作为,实现高效探索与精准推理。全面评估显示,STAR-R1在全部11项指标上达到当前最优,跨视角场景下比SFT高出23%。进一步分析揭示其类人行为,并展现出通过比较所有物体来提升空间推理的独特能力。代码、模型权重与数据将公开于https://github.com/zongzhao23/STAR-R1。

原文摘要 · Abstract (English)

Multimodal Large Language Models (MLLMs) have demonstrated remarkable capabilities across diverse tasks, yet they lag significantly behind humans in spatial reasoning. We investigate this gap through Transformation-Driven Visual Reasoning (TVR), a challenging task requiring identification of object transformations across images under varying viewpoints. While traditional Supervised Fine-Tuning (SFT) fails to generate coherent reasoning paths in cross-view settings, sparse-reward Reinforcement Learning (RL) suffers from inefficient exploration and slow convergence. To address these limitations, we propose STAR-R1, a novel framework that integrates a single-stage RL paradigm with a fine-grained reward mechanism tailored for TVR. Specifically, STAR-R1 rewards partial correctness while penalizing excessive enumeration and passive inaction, enabling efficient exploration and precise reasoning. Comprehensive evaluations demonstrate that STAR-R1 achieves state-of-the-art performance across all 11 metrics, outperforming SFT by 23% in cross-view scenarios. Further analysis reveals STAR-R1's anthropomorphic behavior and highlights its unique ability to compare all objects for improving spatial reasoning. Our work provides critical insights in advancing the research of MLLMs and reasoning models. The codes, model weights, and data will be publicly available at https://github.com/zongzhao23/STAR-R1.

空间推理强化学习多模态大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。