arXiv:2507.19983cs.ROcs.AI2025-07被引 2

用语义关键点实现通用衣物操作,支持多种衣物和任务。

CLASP: General-Purpose Clothes Manipulation with Semantic Keypoints

  • 以语义关键点为中间表示,连接高层规划与底层执行
  • 在多种衣物类型上实现折叠、挂起等任务的高成功率
  • 适合需要通用衣物处理能力的家用机器人研究

衣物操作(如折叠、悬挂)是家庭服务机器人的重要能力。尽管已有进展,但多数方法仍局限于特定衣物类型和任务,因衣物几何结构复杂且维度高。本文提出基于语义关键点的通用衣物操作方法(CLASP),可处理T恤、短裤、裙子、长裙等多种衣物及折叠、平整、悬挂等任务。核心思想是使用‘左袖’‘右肩’等语义关键点作为稀疏的空间-语义表示,从RGB-D图像中可靠提取,有效支撑各类衣物操作策略。CLASP通过语义关键点连接高层任务规划与底层动作执行:高层利用视觉语言模型(VLMs)预测基于关键点的任务计划;低层则调用预建的条件化操作技能执行计划。大量仿真实验表明,CLASP在多种任务和衣物类型上优于现有先进方法,展现优异性能与泛化能力。进一步在Franka双臂系统上对折叠、平整、悬挂、放置四项任务的实机验证,证实其在真实场景中的有效性。

原文摘要 · Abstract (English)

Clothes manipulation, such as folding or hanging, is a critical capability for home service robots. Despite recent advances, most existing methods remain limited to specific clothes types and tasks, due to the complex, high-dimensional geometry of clothes. This paper presents CLothes mAnipulation with Semantic keyPoints (CLASP), which aims at general-purpose clothes manipulation over diverse clothes types, T-shirts, shorts, skirts, long dresses, ..., as well as different tasks, folding, flattening, hanging, .... The core idea of CLASP is semantic keypoints-e.g., ''left sleeve'' and ''right shoulder''-a sparse spatial-semantic representation, salient for both perception and action. Semantic keypoints of clothes can be reliably extracted from RGB-D images and provide an effective representation for a wide range of clothes manipulation policies. CLASP uses semantic keypoints as an intermediate representation to connect high-level task planning and low-level action execution. At the high level, it exploits vision language models (VLMs) to predict task plans over the semantic keypoints. At the low level, it executes the plans with the help of a set of pre-built manipulation skills conditioned on the keypoints. Extensive simulation experiments show that CLASP outperforms state-of-the-art baseline methods on multiple tasks across diverse clothes types, demonstrating strong performance and generalization. Further experiments with a Franka dual-arm system on four distinct tasks-folding, flattening, hanging, and placing-confirm CLASP's performance on real-life clothes manipulation.

衣物操作语义关键点机器人抓取通用控制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。