让机器人仅凭自身感觉完成任务，靠长期记忆补足视觉缺失
ProDapt: Proprioceptive Adaptation using Long-term Memory Diffusion
- 用关键点记录过去与环境的接触，作为动作生成的记忆输入
- 在仿真和真实机械臂上验证，无视觉时也能完成任务
- 适合视觉易失效的太空、水下等高风险场景
扩散模型已革新模仿学习，使机器人能复现复杂行为。但传统扩散模型依赖摄像头等外感受传感器观测环境，缺乏长期记忆。在太空、军事和水下应用中，机器人必须在外部传感器失效时仍能运行，仅依靠本体感受信息。本文提出ProDapt方法，在扩散过程中融入机器人与环境交互的长期记忆，仅使用本体感受数据完成任务。通过识别“关键点”——即重要历史观测，并将其作为策略输入，实现记忆增强。我们在UR10e机械臂上进行了仿真和真实实验，验证了长期记忆对任务完成的必要性。
原文摘要 · Abstract (English)
Diffusion models have revolutionized imitation learning, allowing robots to replicate complex behaviours. However, diffusion often relies on cameras and other exteroceptive sensors to observe the environment and lacks long-term memory. In space, military, and underwater applications, robots must be highly robust to failures in exteroceptive sensors, operating using only proprioceptive information. In this paper, we propose ProDapt, a method of incorporating long-term memory of previous contacts between the robot and the environment in the diffusion process, allowing it to complete tasks using only proprioceptive data. This is achieved by identifying "keypoints", essential past observations maintained as inputs to the policy. We test our approach using a UR10e robotic arm in both simulation and real experiments and demonstrate the necessity of this long-term memory for task completion.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。