用Transformer控制双臂机械手精准对齐不同纹理布料
Transformer Driven Visual Servoing for Fabric Texture Matching Using Dual-Arm Manipulator
- 结合Transformer视觉伺服与双臂阻抗控制，实时调节布料姿态
- 合成数据训练的模型实现零样本部署，真实场景对齐误差小
- 适合需要高精度布料对位的智能制造场景
本文提出一种方法，利用双臂机械手和灰度相机，将一块布料精确对齐并放置在另一块布料之上，使表面纹理完全匹配。所提控制方案融合基于Transformer的视觉伺服与双臂阻抗控制，可在放置过程中同步调节布料姿态并施加张力以保持平整。提出的Transformer网络采用预训练主干和新设计的差异提取注意力模块（DEAM），显著提升姿态差异预测精度。网络仅在渲染软件生成的合成图像上训练，即可实现零样本部署，无需针对特定布料纹理进行实拍训练。真实世界实验表明，该系统能准确对齐不同纹理的布料。
原文摘要 · Abstract (English)
In this paper, we propose a method to align and place a fabric piece on top of another using a dual-arm manipulator and a grayscale camera, so that their surface textures are accurately matched. We propose a novel control scheme that combines Transformer-driven visual servoing with dualarm impedance control. This approach enables the system to simultaneously control the pose of the fabric piece and place it onto the underlying one while applying tension to keep the fabric piece flat. Our transformer-based network incorporates pretrained backbones and a newly introduced Difference Extraction Attention Module (DEAM), which significantly enhances pose difference prediction accuracy. Trained entirely on synthetic images generated using rendering software, the network enables zero-shot deployment in real-world scenarios without requiring prior training on specific fabric textures. Real-world experiments demonstrate that the proposed system accurately aligns fabric pieces with different textures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。