用高斯点云建模颗粒物动态,实现高效智能抓取。
Gaussian Splatting Visual MPC for Granular Media Manipulation
- 基于高斯点云的视觉动态模型,捕捉颗粒堆复杂状态
- 在仿真与真实场景中实现零样本迁移,任务成功率显著提升
- 适合需要精准操控散状物料的机器人应用
近期学习型3D表示的进步推动了刚体物体复杂操作任务的解决,但对豆类、坚果和米粒等颗粒材料的操作仍具挑战,因其粒子间相互作用复杂、状态高维且部分可观测、无法视觉追踪单个颗粒,且精确动力学预测计算量大。现有深度隐式动力学模型因缺乏归纳偏置,在颗粒操作中泛化能力差。本文提出一种新方法:在高斯点云表征的场景上学习视觉动力学模型,并通过模型预测控制实现颗粒材料操纵。该方法能高效优化复杂操作任务。我们在仿真与真实环境中评估,结果表明其可解决未见过的规划任务,并实现零样本迁移。相比现有方法,预测与操作性能均有显著提升。
原文摘要 · Abstract (English)
Recent advancements in learned 3D representations have enabled significant progress in solving complex robotic manipulation tasks, particularly for rigid-body objects. However, manipulating granular materials such as beans, nuts, and rice, remains challenging due to the intricate physics of particle interactions, high-dimensional and partially observable state, inability to visually track individual particles in a pile, and the computational demands of accurate dynamics prediction. Current deep latent dynamics models often struggle to generalize in granular material manipulation due to a lack of inductive biases. In this work, we propose a novel approach that learns a visual dynamics model over Gaussian splatting representations of scenes and leverages this model for manipulating granular media via Model-Predictive Control. Our method enables efficient optimization for complex manipulation tasks on piles of granular media. We evaluate our approach in both simulated and real-world settings, demonstrating its ability to solve unseen planning tasks and generalize to new environments in a zero-shot transfer. We also show significant prediction and manipulation performance improvements compared to existing granular media manipulation methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。