用密集奖励强化场景图推理,让大模型更懂空间关系。
SpatialThinker: Reinforcing Scene Graph-Grounded Spatial Reasoning via Dense Rewards
- 将场景图生成融入推理链,实现端到端空间理解。
- 仅用7000样本训练,性能媲美GPT-5并超越GPT-4o。
- 适合需要精准空间推理的视觉问答任务。
多模态大语言模型在视觉语言任务中取得显著进展,但在空间推理方面仍存挑战。现有方法依赖大规模数据集、显式3D输入、特定架构修改或稀疏强化学习,难以有效引导空间感知。我们提出SpatialThinker,据知是首个通过在线强化学习在单次推理中统一场景图生成(SGG)与视觉推理的MLLM。模型通过构建任务相关物体及其关系的心理场景图,借助密集空间奖励进行类人化空间推理。主要贡献包括:(1) 将场景图生成直接嵌入推理链,而非独立预处理;(2) 构建高质量空间视觉问答数据集STVQA-7K,采用可扩展合成流程;(3) 设计密集空间奖励机制,在强化学习中强化结构化空间锚定,并提升泛化能力。SpatialThinker-7B相较监督微调(SFT)提升3.6倍,分布内/外泛化能力比稀疏强化学习高1.7倍;仅用7000样本训练即达到GPT-5水平,优于GPT-4o;SpatialThinker-30B在14项空间与真实世界基准上平均超越GPT-5与Claude 4 Sonnet,证明结构化空间锚定与奖励对齐推理可在小数据下实现鲁棒空间理解。
原文摘要 · Abstract (English)
Multimodal large language models (MLLMs) have achieved remarkable progress in vision-language tasks, but continue to struggle with spatial reasoning. Existing spatial MLLMs rely on large-scale datasets, explicit 3D inputs, architecture-specific modifications, or sparse Reinforcement Learning (RL) methods that provide insufficient guidance for spatially-grounded reasoning. We introduce SpatialThinker. To our knowledge, it is the first MLLM unifying Scene Graph Generation (SGG) and visual reasoning in a single pass via online RL. The model simulates human-like spatial perception by constructing a mental scene graph of task-relevant objects and relations, and reasoning toward an answer via dense spatial rewards. Our contributions are threefold: (1) SGG-grounded reasoning: integrating SGG directly within the reasoning chain rather than as a disjoint preprocessing step; (2) STVQA-7K: a high-quality spatial VQA training dataset via a scalable synthesis pipeline; and (3) a dense spatial reward design that enforces structured grounding during RL and generalizes to improve broad visual perception. SpatialThinker-7B achieves 3.6$\times$ larger gains over SFT and $1.7\times$ better in- and out-of-distribution generalization than sparse RL. Trained on only 7K samples, SpatialThinker-7B matches GPT-5 and outperforms GPT-4o, while SpatialThinker-30B surpasses both GPT-5 and Claude 4 Sonnet on average across 14 spatial and real-world benchmarks, demonstrating that structured spatial grounding with reward-aligned reasoning enables robust spatial understanding with limited data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。