构建多传感器触觉数据集,评估不同触觉传感器间迁移性能。
TacVerse: A Multi-Sensor Dataset and Benchmark for Cross-Sensor Vision-Based Tactile Perception

- 收集7种触觉传感器的10.68万张图像,支持多种下游任务。
- 跨传感器直接迁移导致性能显著下降,力回归最敏感。
- 自监督预训练提升泛化能力,适合研究数据高效适配方法。
基于视觉的触觉传感器(VBTS)通过内部摄像头捕捉形变来推断接触几何与受力信息,但不同传感器设计间的泛化能力尚不明确。我们提出TacVerse,一个用于跨传感器视觉触觉感知的多传感器数据集与基准测试平台。该数据集包含来自7种VBTS的106,800张触觉图像,支持形状分类、条纹分类和力回归三项下游任务。实验在三种设置下进行:传感器内训练、零样本跨传感器迁移、少样本适应。所有任务在传感器内均表现良好,说明观测信息具有价值;但直接跨传感器迁移导致性能大幅下降,其中力回归最敏感,条纹分类次之,形状分类相对鲁棒。少样本适应可稳定提升力回归在未知传感器上的表现,但无法完全达到传感器内最优水平。表示学习研究进一步表明,掩码自编码器(MAE)预训练在各类任务与传感器中均带来最一致的性能提升。TacVerse为研究传感器偏移、数据高效适应及自监督学习提供了可控实验环境。
原文摘要 · Abstract (English)
Vision-based tactile sensors (VBTSs) enable robots to infer contact geometry and force-related cues by imaging deformation through an internal camera, yet generalisation across sensor designs remains poorly understood. We present TacVerse, a multi-sensor dataset and benchmark for cross-sensor vision-based tactile perception. The dataset contains 106,800 tactile images from seven VBTSs and supports three downstream tasks: shape classification, grating classification, and force regression. Experiments are conducted under three settings: within-sensor training, zero-shot cross-sensor transfer, and few-shot adaptation. Strong within-sensor performance across all tasks indicates that the collected tactile observations are informative for the target objectives. Direct cross-sensor transfer, however, leads to substantial degradation. Shape classification is comparatively robust, whereas grating classification and force regression are more sensitive to sensor shift. Few-shot adaptation for force regression consistently improves performance on unseen target sensors but does not fully close the gap to within-sensor upper bounds. A representation study further shows that MAE (Masked Autoencoder) pretraining provides the most consistent gains across tasks and sensors. TacVerse provides a controlled testbed for studying sensor shift, data-efficient adaptation, and self-supervised learning in tactile perception.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。