用3D点直接注入动作头,显著提升机器人操作的泛化能力
Direct Action-Head Injection of A Grounded 3D Point Unlocks Spatial and Task Generalization

- 将物体位置表示为3D点,直接注入动作头
- 在LIBERO-PRO上成功率提升46.3和32.1点
- 轻量模块无需修改预训练模型
视觉-语言-动作(VLA)模型通过大规模视觉-语言预训练实现灵活的机器人操作,但在测试时仍存在空间泛化与任务泛化脆弱的问题。现有方法依赖语言或视觉提示获取2D坐标,但效果有限。本文提出一种轻量、模型无关的模块:将接地信号以3D点形式表示,计算其相对于夹爪的相对位移,并通过自适应层归一化直接注入动作头。该模块仅由两层MLP构成,无需修改VLA主干或预训练流程。在LIBERO-PRO数据集上,该方法使GR00T-N1.6在任务扰动下成功率从31.2提升至77.5,在位置扰动下从28.1提升至60.2(分别提升46.3和32.1点)。π₀.₅指标也取得类似增益,验证了方法的主干无关性。结果表明,将充分提升的3D接地信号直接注入动作头,是解锁空间与任务泛化的关键。
原文摘要 · Abstract (English)
Vision-Language-Action (VLA) models leverage large-scale vision-language pretraining for flexible robot manipulation, yet at test time they remain brittle along two axes: spatial generalization, when object positions differ from those seen during training, and task generalization, when a familiar scene is paired with a different language instruction than the one seen in training. A growing family of methods addresses this brittleness by endowing a policy with the spatial and task-aware information such as 2D pixel-coordinate for object localization and placement. However, we find that existing representation through language prompting or visual prompting does not address the limitations; in contrast, exploiting a 3D point-based representation and feeding it directly to the action head leads to substantial improvements-revealing that how the grounding signal is represented and injected into the VLA is the true game changer. Thus, we propose a lightweight, model-agnostic module that represents the grounding signal in 3D, computes its relative displacement to the gripper, and injects the resulting spatial embedding directly into the action head through adaptive layer normalization. The entire module is a two-layer MLP that requires no changes to the VLA backbone or pretraining pipeline. On LIBERO-PRO, our method improves the average success rate of GR00T-N1.6 from 31.2 to 77.5 points under task perturbation and from 28.1 to 60.2 points under position perturbation (gains of 46.3 and 32.1 points). Comparable gains are achieved for $π_{0.5}$ as well, demonstrating that the mechanism is backbone-agnostic. Together, these results support our central finding: given adequate grounding lifted into 3D, injecting it directly into the action head is what unlocks both spatial and task generalization in VLAs-achievable with nothing more than a lightweight module on top of a pretrained backbone.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。