用人类示范数据训练机器人，能自动折叠任意皱巴巴的布料。
SSFold: Learning to Fold Arbitrary Crumpled Cloth Using Graph Dynamics from Human Demonstration
- 双流网络：一个决定抓取点，一个用图模型推断被遮挡布料的完整形状。
- 在真实机器人上测试，四种折叠任务成功率最高达99%，最低67%。
- 不依赖仿真，直接从真人示范学动作，适合实际场景中的布料操作。
机器人处理布料面临挑战，因布料动态复杂且配置空间维度高。以往方法多聚焦于单一平整或折叠任务，严重依赖仿真，难以弥合模拟与现实之间的差距。为此，我们提出一种双流架构，包含时序与空间路径，将平整与折叠任务统一为可适配多种布料类型和状态的通用策略模型。时序流确定布料的抓取与放置位置；空间流基于连接动力学模型，从部分点云数据构建可见性图，使机器人能从不完整观测中推断出布料全貌。为缩小模拟到现实的差距，我们采用手部追踪检测算法收集并整合人类示范数据，输入端到端神经网络，提升真实环境适应能力。在UR5机器人上验证了四种不同目标形状的折叠任务，均能从任意褶皱初始状态成功折叠，成功率分别为99%、99%、83%和67%。性能优于现有最先进方法，在未见过的布料（颜色、形状、刚度各异）上也表现出强泛化能力。视频与源码见：https://zcswdt.github.io/SSFold/
原文摘要 · Abstract (English)
Robotic cloth manipulation faces challenges due to the fabric's complex dynamics and the high dimensionality of configuration spaces. Previous methods have largely focused on isolated smoothing or folding tasks and overly reliant on simulations, often failing to bridge the significant sim-to-real gap in deformable object manipulation. To overcome these challenges, we propose a two-stream architecture with sequential and spatial pathways, unifying smoothing and folding tasks into a single adaptable policy model that accommodates various cloth types and states. The sequential stream determines the pick and place positions for the cloth, while the spatial stream, using a connectivity dynamics model, constructs a visibility graph from partial point cloud data of the self-occluded cloth, allowing the robot to infer the cloth's full configuration from incomplete observations. To bridge the sim-to-real gap, we utilize a hand tracking detection algorithm to gather and integrate human demonstration data into our novel end-to-end neural network, improving real-world adaptability. Our method, validated on a UR5 robot across four distinct cloth folding tasks with different goal shapes, consistently achieves folded states from arbitrary crumpled initial configurations, with success rates of 99\%, 99\%, 83\%, and 67\%. It outperforms existing state-of-the-art cloth manipulation techniques and demonstrates strong generalization to unseen cloth with diverse colors, shapes, and stiffness in real-world experiments.Videos and source code are available at: https://zcswdt.github.io/SSFold/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。