零样本实现衣物正反面翻转,仅用单目摄像头部署成功率达81.3%。
Right-Side-Out: Learning Zero-Shot Sim-to-Real Garment Reversal
- 分解任务为拖拽/甩动开洞、插入/拉拽翻面,降低动作空间。
- 模拟器生成海量数据,真实部署零样本成功率最高达81.3%。
- 适合无标注数据、高动态且严重遮挡的物理操作任务研究者。
将衣物翻至正面朝外是一项极具挑战性的操作任务:过程高度动态,接触频繁变化,且存在严重视觉遮挡。我们提出Right-Side-Out,一个零样本模拟到现实的框架,通过利用任务结构有效解决该问题。将任务分解为两个阶段:先通过拖拽/甩动创建并稳定开口,再通过插入/拉拽完成翻转。每个阶段采用基于深度推断的关键点参数化双臂原语,显著压缩动作空间的同时保持鲁棒性。高效的仿真数据生成依赖于自研的高保真GPU并行材料点法(MPM)模拟器,能够建模薄壳形变并高效处理批量回放中的接触问题。基于此模拟器,我们的全自动流水线通过随机化衣物几何、材料参数和视角,生成无需人工标注的深度图、掩码及每原语关键点标签。仅使用单个深度相机,完全在仿真中训练的策略可零样本部署于真实硬件,最高成功率达81.3%。通过任务分解与高保真模拟,本框架实现了对高度动态、严重遮挡任务的攻克,无需繁琐的人工示范。
原文摘要 · Abstract (English)
Turning garments right-side out is a challenging manipulation task: it is highly dynamic, entails rapid contact changes, and is subject to severe visual occlusion. We introduce Right-Side-Out, a zero-shot sim-to-real framework that effectively solves this challenge by exploiting task structures. We decompose the task into Drag/Fling to create and stabilize an access opening, followed by Insert&Pull to invert the garment. Each step uses a depth-inferred, keypoint-parameterized bimanual primitive that sharply reduces the action space while preserving robustness. Efficient data generation is enabled by our custom-built, high-fidelity, GPU-parallel Material Point Method (MPM) simulator that models thin-shell deformation and provides robust and efficient contact handling for batched rollouts. Built on the simulator, our fully automated pipeline scales data generation by randomizing garment geometry, material parameters, and viewpoints, producing depth, masks, and per-primitive keypoint labels without any human annotations. With a single depth camera, policies trained entirely in simulation deploy zero-shot on real hardware, achieving up to 81.3% success rate. By employing task decomposition and high fidelity simulation, our framework enables tackling highly dynamic, severely occluded tasks without laborious human demonstrations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。