让机器人通过触觉与视觉结合，实时识别抓握物体的材质。
SemanticFeels: Semantic Labeling during In-Hand Manipulation
- 用触觉+视觉数据训练CNN，预测局部材质
- 将材质预测嵌入隐式形状网络，实现几何与材质联合建模
- 在多材质物体上达到79.87%平均识别准确率，适合具身智能研究
随着机器人日益融入日常任务，其在抓握过程中感知物体形状与属性的能力对实现自适应智能行为至关重要。我们提出SemanticFeels，是NeuralFeels框架的扩展，将语义标签与神经隐式形状表示相结合，融合视觉与触觉信息。为展示其应用，我们聚焦于材料分类：高分辨率Digit触觉读数通过微调的EfficientNet-B0卷积神经网络（CNN）处理，生成局部材料预测，并嵌入增强型有符号距离场（SDF）网络中，联合预测几何结构与连续材料区域。实验结果表明，该系统在单材料与多材料物体上均实现了预测与实际材料的高度一致，在多材料物体上的多次抓取试验中平均匹配准确率达79.87%。
原文摘要 · Abstract (English)
As robots become increasingly integrated into everyday tasks, their ability to perceive both the shape and properties of objects during in-hand manipulation becomes critical for adaptive and intelligent behavior. We present SemanticFeels, an extension of the NeuralFeels framework that integrates semantic labeling with neural implicit shape representation, from vision and touch. To illustrate its application, we focus on material classification: high-resolution Digit tactile readings are processed by a fine-tuned EfficientNet-B0 convolutional neural network (CNN) to generate local material predictions, which are then embedded into an augmented signed distance field (SDF) network that jointly predicts geometry and continuous material regions. Experimental results show that the system achieves a high correspondence between predicted and actual materials on both single- and multi-material objects, with an average matching accuracy of 79.87% across multiple manipulation trials on a multi-material object.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。