arXiv:2608.01083cs.RO2026-08

用稀疏关键点+稠密对应关系,让机器人更好抓捏变形物体。

Sparse Meets Dense: Correspondence Guided Robotic Manipulation with Rigid-Deformable Interactions

论文配图:Sparse Meets Dense: Correspondence Guided Robotic Manipulation with Rigid-Deformable Interactions
图 1 · 摘自论文原文
  • 设计稀疏关键点捕捉结构、任务和接触信息
  • 结合稠密对应实现关键点精准追踪与新形态泛化
  • 仅需少量示范即可迁移至新任务,适合复杂操作场景

涉及刚体-柔性体交互的操作(如挂衣服、穿衣服)在日常生活中常见,对家用机器人至关重要。相比单物体或刚体间交互,这类任务因多点接触和柔性体复杂动力学而更具挑战性,传统以6D姿态或结构点为主的物体中心表征难以满足需求。本文提出一种面向刚体-柔性体交互的混合对应表征方法:首先生成结构、任务和交互感知的稀疏关键点,基于刚体与柔性体全局结构并由局部接触过滤;为解决柔性体高维动态下关键点追踪难题,进一步在柔性体上构建稠密对应关系,实现精确追踪。该设计融合稀疏关键点的任务特异性与稠密对应的关系泛化能力,支持仅凭少量示范的一次性任务迁移。大量实验验证了方法的有效性与广泛适用性。

原文摘要 · Abstract (English)

Manipulation involving rigid-deformable interactions, such as hanging clothes or dressing humans, is common in daily life, making it essential for household robots. Compared to single-object manipulation or interactions between rigid bodies, these tasks are particularly challenging due to the rich multi-point contacts and the complex dynamics of the deformable bodies during interaction. Therefore, object-centric representations such as 6D poses or structural points without task-specific information become insufficient for these interactions. In this work, we propose a hybrid correspondence-based representation tailored for rigid-deformable interactions. First, to capture intricate interaction information, we introduce structure-, task-, and interaction-aware sparse keypoints. The keypoints are generated based on the global structures of both rigid and deformable objects, and filtered by their local interaction contacts. However, tracking these sparse keypoints through the interaction remains difficult due to the high-dimensional dynamics of deformable objects. Therefore, we further construct dense correspondences on the deformable objects for accurate keypoint tracking throughout the manipulation. This hybrid design combines the advantages of both representations: sparse keypoints encode rich, task-specific information for fine-grained manipulation, while dense correspondences ensure efficient tracking and generalization to novel deformations, shapes, and scenarios. Together, they enable one-shot transfer to new tasks with minimal demonstrations. Extensive experiments demonstrate the effectiveness and broad applicability of our method.

机器人操作刚体-柔性体对应关系少样本迁移

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。