arXiv:2505.11032cs.ROcs.AI2025-05NeurIPS被引 20

构建首个可复用的精细衣物操作环境,实现跨形态衣物的通用操控。

DexGarmentLab: Dexterous Garment Manipulation Environment with Generalizable Policy

  • 基于衣物结构对应性,仅需一次专家示范自动生成多样化轨迹数据。
  • 提出分层策略HALO,在未见衣物上成功完成复杂操作任务。
  • 适用于机器人服装抓取、智能穿搭等需要高精度操作的场景。

衣物操作因品类多样、几何形状和形变复杂而极具挑战,尽管人类凭借灵巧双手能轻松处理,现有研究却难以复现该能力,主要受限于缺乏真实感的精细衣物操作仿真环境。为此,我们提出DexGarmentLab,首个专为精细(尤其是双臂)衣物操作设计的环境,包含15个任务场景的大规模高质量3D资产,并优化了针对衣物建模的仿真技术以缩小仿真到现实的差距。以往数据采集依赖遥操作或训练专家强化学习策略,成本高且效率低。本文利用衣物结构对应性,仅需单次专家示范即可自动生成多样化轨迹数据集,大幅减少人工干预。然而,即使大量示范也无法覆盖衣物的所有状态,因此亟需新算法。为提升对不同衣物形状与形变的泛化能力,我们提出分层衣物操作策略HALO:先识别可迁移的可操作点精确定位操作区域,再生成通用轨迹完成任务。通过大量实验与对比分析,验证HALO持续优于现有方法,能在显著形状与形变差异下成功推广至未见过的衣物实例,而其他方法则失败。项目主页:https://wayrise.github.io/DexGarmentLab/

原文摘要 · Abstract (English)

Garment manipulation is a critical challenge due to the diversity in garment categories, geometries, and deformations. Despite this, humans can effortlessly handle garments, thanks to the dexterity of our hands. However, existing research in the field has struggled to replicate this level of dexterity, primarily hindered by the lack of realistic simulations of dexterous garment manipulation. Therefore, we propose DexGarmentLab, the first environment specifically designed for dexterous (especially bimanual) garment manipulation, which features large-scale high-quality 3D assets for 15 task scenarios, and refines simulation techniques tailored for garment modeling to reduce the sim-to-real gap. Previous data collection typically relies on teleoperation or training expert reinforcement learning (RL) policies, which are labor-intensive and inefficient. In this paper, we leverage garment structural correspondence to automatically generate a dataset with diverse trajectories using only a single expert demonstration, significantly reducing manual intervention. However, even extensive demonstrations cannot cover the infinite states of garments, which necessitates the exploration of new algorithms. To improve generalization across diverse garment shapes and deformations, we propose a Hierarchical gArment-manipuLation pOlicy (HALO). It first identifies transferable affordance points to accurately locate the manipulation area, then generates generalizable trajectories to complete the task. Through extensive experiments and detailed analysis of our method and baseline, we demonstrate that HALO consistently outperforms existing methods, successfully generalizing to previously unseen instances even with significant variations in shape and deformation where others fail. Our project page is available at: https://wayrise.github.io/DexGarmentLab/.

机器人操作衣物生成强化学习仿真环境

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。