构建复杂可动物体操控环境,实现基于视觉的自适应抓取策略学习。
AdaManip: Adaptive Articulated Object Manipulation Environments and Policy Learning
- 设计9类复杂可动物体环境,支持多阶段状态依赖操作
- 提出3D视觉扩散模型驱动的模仿学习框架,实现自适应策略生成
- 在仿真与真实场景中验证方法有效性,适用于具状态依赖的复杂任务
可动物体操控是机器人在现实场景中执行多样化任务的关键能力。这类物体由通过关节连接的多个部件构成,其功能机制依赖于复杂的相对运动。例如,保险箱包含门、把手和锁,只有在锁未上锁时门才能打开。内部结构(如锁的状态或关节角度约束)无法仅通过视觉直接观测,因此成功操控需基于试错进行自适应调整,而非一次性视觉推断。然而,以往的数据集与仿真环境主要关注简单操控机制,能从外观直接推断完整操作过程。为提升自适应操控机制的多样性和复杂性,我们构建了一个新型可动物体操控环境,包含9类物体。基于该环境,我们进一步提出自适应示范收集方法与基于3D视觉扩散的模仿学习流程,以学习自适应操控策略。通过仿真与真实世界实验验证了所提设计与方法的有效性。
原文摘要 · Abstract (English)
Articulated object manipulation is a critical capability for robots to perform various tasks in real-world scenarios. Composed of multiple parts connected by joints, articulated objects are endowed with diverse functional mechanisms through complex relative motions. For example, a safe consists of a door, a handle, and a lock, where the door can only be opened when the latch is unlocked. The internal structure, such as the state of a lock or joint angle constraints, cannot be directly observed from visual observation. Consequently, successful manipulation of these objects requires adaptive adjustment based on trial and error rather than a one-time visual inference. However, previous datasets and simulation environments for articulated objects have primarily focused on simple manipulation mechanisms where the complete manipulation process can be inferred from the object's appearance. To enhance the diversity and complexity of adaptive manipulation mechanisms, we build a novel articulated object manipulation environment and equip it with 9 categories of objects. Based on the environment and objects, we further propose an adaptive demonstration collection and 3D visual diffusion-based imitation learning pipeline that learns the adaptive manipulation policy. The effectiveness of our designs and proposed method is validated through both simulation and real-world experiments. Our project page is available at: https://adamanip.github.io
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。