构建机器人采集的触觉视觉语言数据集，推动未知材质触觉泛化研究
RCT: A Robot-Collected Touch-Vision-Language Dataset for Tactile Generalization
- 用机器人对122种工业材料进行多位置按压，采集29,279帧触觉数据
- 去除接触序列重叠后，触觉文本匹配准确率下降17.7个百分点，凸显泛化挑战
- 适合关注机器人触觉感知、跨材质泛化与数据评估方法的研究者
针对机器人在开放世界中操作物体时触觉表征需泛化至未见材料的问题，我们提出RCT（Robotic Contact Tactile）数据集，该数据集由机器人在122种工业参考材料（分属7类）上进行多位置按压采集，共获得29,279帧触觉数据，使用三个DIGIT传感器记录。每个按压过程被完整保留为接触序列，支持跨材料、类别、传感器、接触位置和接触序列的留出评估。单次按压内的帧高度相关：若采用随机帧划分，训练与测试集可能包含同一物理交互的近似重复样本。固定编码器时，消除接触序列重叠使触觉到文本的Recall@1降低17.7个百分点；当训练时额外留出材料，性能急剧下降，留出材料的Recall@1平均仅为25.1% ± 6.1%（三次留出实验均值）。公开的TVL/HCT划分同样存在此问题：每个测试接触序列均出现在训练集中，且原始像素最近邻可复现正确序列的比例达98.3%。均匀采样单次按压能改善对比学习效果，基于RCT训练的嵌入在未见材料上提升类别探测性能。RCT使接触序列感知的留出材料评估具备可复现性，并揭示了新材料泛化作为机器人触觉感知的核心挑战。数据集已开源：https://faerber-lab.github.io/RCT/
原文摘要 · Abstract (English)
For robots manipulating open-world objects, tactile representations must generalize to unseen materials. We introduce RCT (Robotic Contact Tactile), a robot-collected touch-vision-language dataset with 29,279 tactile frames from full robot presses on 122 industrial reference materials in 7 categories, recorded with three DIGIT sensors at multiple contact positions. RCT preserves each press as a contact sequence, enabling held-out evaluation across materials, categories, sensors, contact positions, and contact sequences. Frames from one press are strongly correlated: frame-random splits can place near-duplicate observations of the same physical interaction in both training and test. With the encoder held fixed, removing contact-sequence overlap reduces tactile-to-text Recall@1 by 17.7 percentage points. When materials are additionally held out at training time, performance drops sharply, leaving held-out-material Recall@1 at 25.1 +/- 6.1% averaged over three held-out draws. The public TVL/HCT split shows the same structure: every test contact sequence appears in training, and raw-pixel nearest neighbors recover the correct sequence in 98.3% of cases. Uniformly sampling a press improves contrastive training, and RCT-trained embeddings improve category probes on unseen materials. RCT makes contact-sequence-aware, held-out-material evaluation reproducible and exposes novel-material generalization as a central challenge for robotic tactile perception. The RCT dataset is open-sourced at https://faerber-lab.github.io/RCT/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。