用生成式扩散模型同时解决布料状态估计与动力学预测难题。
Diffusion Dynamics Models with Generative State Estimation for Cloth Manipulation
- 基于扩散模型，从部分观测重建完整布料状态
- 长时序预测误差降低一个数量级，优于以往方法
- 适合需要复杂变形物体操作的机器人系统应用
布料操作因高度复杂的动态特性、近乎无限的自由度以及频繁自遮挡而极具挑战，这使得状态估计和动力学建模均难以实现。受生成模型近期进展启发，我们假设这些表达能力强的模型能从数据中有效捕捉复杂的布料构型与形变模式。为此，我们提出一种基于扩散的生成方法，同时用于感知与动力学建模。具体地，将状态估计建模为从部分观测重建完整布料状态，将动力学建模建模为在当前状态和机器人动作下预测未来状态。利用基于Transformer的扩散模型，我们的方法实现了高精度的状态重建，并使长时序动力学预测误差相比先前方法降低一个数量级。我们将该动力学模型集成到模型预测控制框架中，在真实机器人系统上成功实现了布料折叠任务，验证了生成模型在部分可观测与复杂动态条件下对可变形物体操作的潜力。
原文摘要 · Abstract (English)
Cloth manipulation is challenging due to its highly complex dynamics, near-infinite degrees of freedom, and frequent self-occlusions, which complicate both state estimation and dynamics modeling. Inspired by recent advances in generative models, we hypothesize that these expressive models can effectively capture intricate cloth configurations and deformation patterns from data. Therefore, we propose a diffusion-based generative approach for both perception and dynamics modeling. Specifically, we formulate state estimation as reconstructing full cloth states from partial observations and dynamics modeling as predicting future states given the current state and robot actions. Leveraging a transformer-based diffusion model, our method achieves accurate state reconstruction and reduces long-horizon dynamics prediction errors by an order of magnitude compared to prior approaches. We integrate our dynamics models with model predictive control and show that our framework enables effective cloth folding on real robotic systems, demonstrating the potential of generative models for deformable object manipulation under partial observability and complex dynamics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。