arXiv:2501.15830cs.ROcs.AI2025-01被引 514

提出空间感知的机器人通用策略模型,提升跨场景操控能力。

SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Model

论文配图:SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Model
图 1 · 摘自论文原文
  • 引入3D位置编码与自适应动作网格,增强空间理解
  • 在110万真实机器人数据上预训练,零样本完成多任务操控
  • 支持新环境快速微调,具备强泛化与迁移能力

本文认为空间理解是机器人操作的核心,提出SpatialVLA模型探索有效的空间表征以构建机器人基础模型。具体而言,引入Ego3D位置编码将3D信息注入视觉-语言-动作模型输入,并提出自适应动作网格,通过可变离散化动作格子表示机器人运动行为,促进跨机器人控制中通用空间动作知识的学习。SpatialVLA首先在包含110万真实世界机器人轨迹的数据集上进行预训练,学习跨多种环境与任务的通用操作策略。预训练后,该模型可直接零样本执行大量任务。仿真与真实机器人上的优异表现验证了其推断复杂运动轨迹及强域内多任务泛化能力。进一步实验表明,自适应动作网格为模型提供了高效微调方式:在新仿真或真实场景中,可重离散化预训练的动作网格,以捕捉特定机器人的空间动作特征。广泛评估结果证明其出色的域内泛化与域外适应能力,凸显空间感知表征在通用机器人策略学习中的关键作用。所有细节与代码将开源。

原文摘要 · Abstract (English)

In this paper, we claim that spatial understanding is the keypoint in robot manipulation, and propose SpatialVLA to explore effective spatial representations for the robot foundation model. Specifically, we introduce Ego3D Position Encoding to inject 3D information into the input observations of the visual-language-action model, and propose Adaptive Action Grids to represent spatial robot movement actions with adaptive discretized action grids, facilitating learning generalizable and transferrable spatial action knowledge for cross-robot control. SpatialVLA is first pre-trained on top of a vision-language model with 1.1 Million real-world robot episodes, to learn a generalist manipulation policy across multiple robot environments and tasks. After pre-training, SpatialVLA is directly applied to perform numerous tasks in a zero-shot manner. The superior results in both simulation and real-world robots demonstrate its advantage of inferring complex robot motion trajectories and its strong in-domain multi-task generalization ability. We further show the proposed Adaptive Action Grids offer a new and effective way to fine-tune the pre-trained SpatialVLA model for new simulation and real-world setups, where the pre-learned action grids are re-discretized to capture robot-specific spatial action movements of new setups. The superior results from extensive evaluations demonstrate the exceptional in-distribution generalization and out-of-distribution adaptation capability, highlighting the crucial benefit of the proposed spatial-aware representations for generalist robot policy learning. All the details and codes will be open-sourced.

机器人学习空间表征通用策略零样本

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。