arXiv:2603.10609cs.RO2026-03

单臂实现布料双手操作,靠视觉触觉融合提升精度与稳定性。

Learning Bimanual Cloth Manipulation with Vision-based Tactile Sensing via Single Robotic Arm

  • 单臂通过新型夹持器实现布料在夹内滑动控制。
  • 边缘识别准确率达96%,定位误差小于1毫米,方向误差4.5°。
  • 结合合成数据生成,减少人工标注,适合低成本布料操作场景。

机器人处理布料仍具挑战,源于布料状态空间高维、易变形及频繁遮挡导致视觉感知受限。尽管双臂系统可缓解部分问题,但会增加硬件与控制复杂度。本文提出Touch G.O.G.,一种紧凑的基于视觉的触觉夹持器及其感知/控制框架,支持单臂双人式布料操作。该框架包含三部分:(1) 创新夹持器设计与控制策略,实现单臂对夹持内布料的滑动控制;(2) 基于视觉基础模型的Vision Transformer流水线(PC-Net用于布料区域分类,PE-Net用于边缘位姿估计),利用真实与合成触觉图像进行训练;(3) 编码器-解码器结构的合成数据生成器(SD-Net),显著降低人工标注需求,生成高保真触觉图像。实验显示,边缘、角点、内部区域及抓取失败的区分准确率达96%,边缘定位误差低于1毫米,方向误差为4.5°。真实世界结果表明,仅用单臂即可可靠展开褶皱布料。这些成果证明Touch G.O.G.是柔性物体操作的一种紧凑且经济高效的解决方案。

原文摘要 · Abstract (English)

Robotic cloth manipulation remains challenging due to the high-dimensional state space of fabrics, their deformable nature, and frequent occlusions that limit vision-based sensing. Although dual-arm systems can mitigate some of these issues, they increase hardware and control complexity. This paper presents Touch G.O.G., a compact vision-based tactile gripper and perception/control framework for single-arm bimanual cloth manipulation. The proposed framework combines three key components: (1) a novel gripper design and control strategy for in-gripper cloth sliding with a single robot arm, (2) a Vision Foundation Model-backboned Vision Transformer pipeline for cloth part classification (PC-Net) and edge pose estimation (PE-Net) using real and synthetic tactile images, and (3) an encoder-decoder synthetic data generator (SD-Net) that reduces manual annotation by producing high-fidelity tactile images. Experiments show 96% accuracy in distinguishing edges, corners, interior regions, and grasp failures, together with sub-millimeter edge localization and 4.5° orientation error. Real-world results demonstrate reliable cloth unfolding, even for crumpled fabrics, using only a single robotic arm. These results highlight Touch G.O.G. as a compact and cost-effective solution for deformable object manipulation.

布料操作触觉感知单臂控制合成数据

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。