arXiv:2512.16237cs.AI2025-12被引 2

用程序生成数据提升大模型空间推理能力

Scaling Spatial Reasoning in MLLMs through Programmatic Data Synthesis

  • 将空间问题转为可执行代码,通过模拟器验证真值
  • 构建300k+图文对数据集,覆盖11000+场景
  • 适合研究视觉语言模型与具身智能的开发者

具身智能是人工智能的重大挑战,其核心瓶颈在于现有模型的空间理解与推理能力有限。当前提升视觉语言模型(VLMs)的方法陷入两难:模板数据集可扩展但结构僵化,人工标注语义多样却不可扩展且计算不精确。我们提出SPRITE框架,利用模拟器与大模型程序化生成可扩展、多样且高质量的空间推理数据。核心创新是将真值生成重构为代码生成任务:用LLMs将复杂空间问题编译为可执行程序,并基于模拟器提取的高精度场景元信息进行验证,确保真值在计算上精确且可验证,同时借助LLM生成丰富的语言表达。基于该流程,我们构建了涵盖3个模拟器、11000+场景和300000+图像/视频指令微调对的数据集。实验证明,基于该数据训练的VLM在多个空间推理基准上显著提升,优于同等规模的其他开源数据集。可扩展性分析进一步验证,克服传统模板方法低多样性问题是构建鲁棒通用空间智能的关键。SPRITE框架代码及完整数据集将公开,以推动空间智能研究。

原文摘要 · Abstract (English)

Embodied intelligence, a grand challenge in artificial intelligence, is fundamentally constrained by the limited spatial understanding and reasoning capabilities of current models. Prevailing efforts to address this through enhancing Vision-Language Models (VLMs) are trapped in a dilemma: template-based datasets are scalable but structurally rigid, while manual annotation is linguistically diverse but unscalable and, critically, computationally imprecise. We introduce SPRITE, a novel framework that overcomes this dilemma by leveraging simulators and large models to programmatically synthesize scalable, diverse, and high-quality spatial reasoning data. The core innovation of SPRITE is to reframe ground-truth generation as a code-generation task. We utilize LLMs to compile complex spatial questions into executable programs, which are then verified against high-precision scene meta-information extracted from simulators. This ensures our ground truth is both computationally precise and verifiable, while the generative power of LLMs provides vast linguistic diversity. Leveraging this pipeline, we have curated a dataset encompassing 3 simulators, 11k+ scenes, and 300k+ image/video instruction-tuning pairs. We demonstrate that a VLM trained on our data achieves significant performance gains on multiple spatial benchmarks and outperforms other open-source datasets of equivalent size. Furthermore, a scalability analysis confirms our hypothesis that overcoming the low-diversity nature of traditional template methods is essential for building robust, generalizable spatial intelligence. We will make the SPRITE framework code and the full 300k+ dataset publicly available to facilitate future research in spatial intelligence.

空间推理数据合成视觉语言模型具身智能

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。