用结构化提示和强化学习提升视觉语言模型的空间推理能力
Enhancing Spatial Reasoning in Vision-Language Models via Chain-of-Thought Prompting and Reinforcement Learning
- 采用场景图引导的分步提示，替代简单思维链，显著提升推理准确率
- 在SAT数据集上用GRPO微调后，通过率比监督微调高12.3%，且对分布外样本更鲁棒
- 适合关注模型泛化性与可解释性的研究人员，尤其关注空间关系理解任务
本研究通过思维链提示和强化学习探究视觉语言模型（VLMs）的空间推理能力。我们评估了不同提示策略的影响，发现简单的思维链格式不仅无效，反而损害模型原生性能。相比之下，基于场景图的结构化多阶段提示（SceneGraph CoT）显著提升空间推理准确率。为进一步提升能力，我们在SAT数据集上使用组相对策略优化（GRPO）对模型进行微调，并在CVBench上评估其表现。相较于监督微调（SFT），GRPO在Pass@1评测中取得更高准确率，且在分布外（OOD）条件下表现出更强鲁棒性。特别地，SFT会过度拟合表面语言模式，当测试时表述变化（如从“更近”变为“更远”）时性能下降；而GRPO能更可靠地泛化，保持稳定表现。研究揭示了强化学习与结构化提示如何改善现代VLMs的空间推理与泛化行为。所有代码开源：https://github.com/Yvonne511/spatial-vlm-investigator
原文摘要 · Abstract (English)
This study investigates the spatial reasoning capabilities of vision-language models (VLMs) through Chain-of-Thought (CoT) prompting and reinforcement learning. We begin by evaluating the impact of different prompting strategies and find that simple CoT formats, where the model generates a reasoning step before the answer, not only fail to help, but can even harm the model's original performance. In contrast, structured multi-stage prompting based on scene graphs (SceneGraph CoT) significantly improves spatial reasoning accuracy. Furthermore, to improve spatial reasoning ability, we fine-tune models using Group Relative Policy Optimization (GRPO) on the SAT dataset and evaluate their performance on CVBench. Compared to supervised fine-tuning (SFT), GRPO achieves higher accuracy on Pass@1 evaluations and demonstrates superior robustness under out-of-distribution (OOD) conditions. In particular, we find that SFT overfits to surface-level linguistic patterns and may degrade performance when test-time phrasing changes (e.g., from "closer to" to "farther from"). GRPO, on the other hand, generalizes more reliably and maintains stable performance under such shifts. Our findings provide insights into how reinforcement learning and structured prompting improve the spatial reasoning capabilities and generalization behavior of modern VLMs. All code is open source at: https://github.com/Yvonne511/spatial-vlm-investigator
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。