arXiv:2410.11402cs.RO2024-10TPAMI被引 49

用扩散模型生成3D场景下移动操作的协调动作轨迹,提升真实机器人执行效果。

M2Diffuser: Diffusion-based Trajectory Optimization for Mobile Manipulation in 3D Scenes

  • 基于3D扫描的场景条件扩散模型,直接生成全身运动轨迹。
  • 在去噪过程中动态优化物理约束,减少执行错误。
  • 在20多个场景中验证,可成功部署到真实机器人上。

扩散模型的进展为具身智能与机器人研究开辟了新路径。尽管复杂机器人行走和技能已取得显著成果,但需协调导航与操作的移动操作仍面临挑战,主要源于高维动作空间、长轨迹及环境交互。本文提出M2Diffuser,一种基于场景条件的扩散生成模型,可基于机器人中心的3D扫描直接生成协调高效的全身体运动轨迹。该模型首先从专家规划器提供的移动操作轨迹中学习轨迹分布,并引入可微优化模块,在推理阶段灵活融合物理约束与任务目标(以代价和能量函数建模),实现每步去噪过程中的物理违规最小化。在超过20个场景的三类移动操作任务上评估表明,M2Diffuser优于现有神经规划方法,并成功将生成轨迹迁移至真实机器人。结果凸显生成式AI对传统规划与学习方法泛化能力的增强潜力,同时强调物理约束对安全可靠执行的关键作用。

原文摘要 · Abstract (English)

Recent advances in diffusion models have opened new avenues for research into embodied AI agents and robotics. Despite significant achievements in complex robotic locomotion and skills, mobile manipulation-a capability that requires the coordination of navigation and manipulation-remains a challenge for generative AI techniques. This is primarily due to the high-dimensional action space, extended motion trajectories, and interactions with the surrounding environment. In this paper, we introduce M2Diffuser, a diffusion-based, scene-conditioned generative model that directly generates coordinated and efficient whole-body motion trajectories for mobile manipulation based on robot-centric 3D scans. M2Diffuser first learns trajectory-level distributions from mobile manipulation trajectories provided by an expert planner. Crucially, it incorporates an optimization module that can flexibly accommodate physical constraints and task objectives, modeled as cost and energy functions, during the inference process. This enables the reduction of physical violations and execution errors at each denoising step in a fully differentiable manner. Through benchmarking on three types of mobile manipulation tasks across over 20 scenes, we demonstrate that M2Diffuser outperforms state-of-the-art neural planners and successfully transfers the generated trajectories to a real-world robot. Our evaluations underscore the potential of generative AI to enhance the generalization of traditional planning and learning-based robotic methods, while also highlighting the critical role of enforcing physical constraints for safe and robust execution.

移动操作扩散模型轨迹优化机器人

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。