用3D体素表示提升机器人操作策略的视觉空间对齐能力
VolumeDP: Modeling Volumetric Representation for Manipulation Policy Learning
- 通过交叉注意力将2D图像特征升维至3D体素空间
- 在LIBERO模拟基准上达成88.8%成功率，领先基线14.8%
- 适合需要强空间推理的机器人抓取与操作任务
模仿学习是机器人操作的重要范式。然而现有视觉模仿方法将2D图像观测直接映射为3D动作输出，造成2D-3D错配，阻碍空间推理并降低鲁棒性。我们提出VolumeDP，一种通过显式3D推理恢复空间对齐的策略架构。VolumeDP首先通过交叉注意力将图像特征提升至体素表示；随后用可学习模块选取任务相关体素，并转换为紧凑的空间令牌集，显著降低计算量同时保留关键几何信息；最后通过多令牌解码器基于完整令牌集预测动作，避免将多个空间令牌损失性聚合为单一描述符。在LIBERO仿真基准上，VolumeDP实现88.8%的平均成功率，优于最强基线14.8%。在ManiSkill和LIBERO-Plus基准上也取得显著提升。真实世界实验进一步验证其在新空间布局、相机视角和环境背景下的高成功率与强泛化能力。代码与视频见项目页：https://yzc0731.github.io/VolumeDP/
原文摘要 · Abstract (English)
Imitation learning is a prominent paradigm for robotic manipulation. However, existing visual imitation methods map 2D image observations directly to 3D action outputs, imposing a 2D-3D mismatch that hinders spatial reasoning and degrades robustness. We present VolumeDP, a policy architecture that restores spatial alignment by explicitly reasoning in 3D. VolumeDP first lifts image features into a Volumetric Representation via cross-attention. It then selects task-relevant voxels with a learnable module and converts them into a compact set of spatial tokens, markedly reducing computation while preserving action-critical geometry. Finally, a multi-token decoder conditions on the entire token set to predict actions, thereby avoiding lossy aggregation that collapses multiple spatial tokens into a single descriptor. VolumeDP achieves a state-of-the-art average success rate of 88.8% on the LIBERO simulation benchmark, outperforming the strongest baseline by a substantial 14.8% improvement. It also delivers large performance gains over prior methods on the ManiSkill and LIBERO-Plus benchmarks. Real-world experiments further demonstrate higher success rates and robust generalization to novel spatial layouts, camera viewpoints, and environment backgrounds. Code and videos are available on the project page: https://yzc0731.github.io/VolumeDP/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。