让视觉语言模型学会空间引导动作，提升机器人任务泛化能力
ST4VLA: Spatially Guided Training for Vision-Language-Action Models
- 用空间提示训练模型，让动作生成与视觉空间先验对齐
- 在谷歌机器人和WidowX上性能从66.1/54.7提升至84.6/73.2
- 适合需要空间理解的机器人动作规划与跨场景泛化任务
大型视觉语言模型（VLM）在多模态理解上表现优异，但在具身任务中难以将指令转化为低层运动动作。我们提出ST4VLA，一种双系统视觉-语言-动作框架，通过空间引导训练使动作学习与VLM中的空间先验对齐。该框架包含两个阶段：(i) 空间定位预训练，利用网络规模和机器人专用数据进行点、框、轨迹预测，赋予VLM可迁移的空间先验；(ii) 空间引导的动作后训练，通过空间提示鼓励模型生成更丰富的空间先验以指导动作生成。此设计保持策略学习中的空间定位能力，并促进空间与动作目标的一致优化。实验表明，相较于原始VLA，ST4VLA在Google Robot上性能从66.1提升至84.6，在WidowX Robot上从54.7提升至73.2，在SimplerEnv上达到新基准。模型还展现出对未见物体、改写指令的更强泛化能力，以及在真实环境中对长时程扰动的鲁棒性。结果表明，可扩展的空间引导训练是实现稳健、通用机器人学习的有前景方向。源码、数据与模型已开源。
原文摘要 · Abstract (English)
Large vision-language models (VLMs) excel at multimodal understanding but fall short when extended to embodied tasks, where instructions must be transformed into low-level motor actions. We introduce ST4VLA, a dual-system Vision-Language-Action framework that leverages Spatial Guided Training to align action learning with spatial priors in VLMs. ST4VLA includes two stages: (i) spatial grounding pre-training, which equips the VLM with transferable priors via scalable point, box, and trajectory prediction from both web-scale and robot-specific data, and (ii) spatially guided action post-training, which encourages the model to produce richer spatial priors to guide action generation via spatial prompting. This design preserves spatial grounding during policy learning and promotes consistent optimization across spatial and action objectives. Empirically, ST4VLA achieves substantial improvements over vanilla VLA, with performance increasing from 66.1 -> 84.6 on Google Robot and from 54.7 -> 73.2 on WidowX Robot, establishing new state-of-the-art results on SimplerEnv. It also demonstrates stronger generalization to unseen objects and paraphrased instructions, as well as robustness to long-horizon perturbations in real-world settings. These results highlight scalable spatially guided training as a promising direction for robust, generalizable robot learning. Source code, data and models are released at https://internrobotics.github.io/internvla-m1.github.io/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。