arXiv:2409.09870cs.RO2024-09ICRA被引 13

用图像翻译提升触觉传感器力预测,跨光照与标记条件通用。

TransForce: Transferable Force Prediction for Vision-based Tactile Sensors with Sequential Image Translation

  • 通过序列图像翻译,将源传感器数据迁移到新传感器场景。
  • 剪切方向误差最低达0.69N(占全范围5.8%),优于单图模型。
  • 纯标记模态更利于剪切力预测,RGB模态在法向更优,适合迁移应用。

基于视觉的触觉传感器(VBTS)提供高分辨率触觉图像,对机器人抓握操作至关重要。然而,由于配对触觉图像与力标签的采集成本高、耗时长,现有方法难以充分利用其力感知能力。本文提出可迁移的力预测模型TransForce,旨在利用已有图像-力配对数据,适应新传感器在不同光照颜色和标记图案下的情况,并显著提升力预测精度,尤其在剪切方向。该模型通过有效实现源域到目标域的触觉图像转换,生成符合新传感器光照与标记特征的图像,同时准确保留原有弹性体形变信息,从而支持更高精度的力估计。采用基于生成序列触觉图像与已有力标签训练的循环力预测模型,在新传感器上实现最低平均误差:x轴0.69N(全量程5.8%)、y轴0.70N(5.8%)、z轴1.11N(6.9%),优于仅使用单张图像的模型。实验还表明,纯标记模态在提升剪切力预测精度方面优于RGB模态,而RGB模态在法向力预测中表现更佳。

原文摘要 · Abstract (English)

Vision-based tactile sensors (VBTSs) provide high-resolution tactile images crucial for robot in-hand manipulation. However, force sensing in VBTSs is underutilized due to the costly and time-intensive process of acquiring paired tactile images and force labels. In this study, we introduce a transferable force prediction model, TransForce, designed to leverage collected image-force paired data for new sensors under varying illumination colors and marker patterns while improving the accuracy of predicted forces, especially in the shear direction. Our model effectively achieves translation of tactile images from the source domain to the target domain, ensuring that the generated tactile images reflect the illumination colors and marker patterns of the new sensors while accurately aligning the elastomer deformation observed in existing sensors, which is beneficial to force prediction of new sensors. As such, a recurrent force prediction model trained with generated sequential tactile images and existing force labels is employed to estimate higher-accuracy forces for new sensors with lowest average errors of 0.69N (5.8\% in full work range) in $x$-axis, 0.70N (5.8\%) in $y$-axis, and 1.11N (6.9\%) in $z$-axis compared with models trained with single images. The experimental results also reveal that pure marker modality is more helpful than the RGB modality in improving the accuracy of force in the shear direction, while the RGB modality show better performance in the normal direction.

触觉传感图像翻译力预测迁移学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。