用毫米波与声学融合识别材质，无视物体形状方向变化
GaMi: Geometry-Agnostic Material Identification via Cross-Modal Subtractive Disentanglement
- 通过跨模态减法解耦，剥离几何干扰提取材质本质特征
- 在20种材料上达95.2%准确率，未见几何条件下超越单模态方法
- 支持少样本跨设备泛化，适合智能机器人等实际场景
非接触式材质识别可支持具身智能的自适应交互，但受几何变化（如朝向、形状、距离）和单模态模糊性影响。本文提出GaMi系统，融合毫米波与声学传感，在无约束几何条件下实现鲁棒识别。基于共位双模传感器间的共享几何一致性，采用样本内跨模态减法解耦框架，语义对齐后减去共享几何上下文，从而分离出内在材质特征。此外，引入样本间对比学习以校正跨模态错位导致的残余干扰。还设计了双模态配对适配策略，实现跨设备少样本泛化。在20种材料上的大量实验表明，GaMi达到95.2%准确率，优于单模态基线在未见几何条件下的表现。
原文摘要 · Abstract (English)
Non-contact material identification enables adaptive interaction for embodied intelligence yet faces challenges from geometry-induced variations (e.g., orientation, shape, distance) and single-modality ambiguities. In this paper, we present GaMi, a multimodal material identification system integrating mmWave and acoustic sensing to robustly operate under unconstrained geometric conditions. By leveraging the insight of shared geometric consistency between co-located bimodal sensors, GaMi employs an intra-sample cross-modal subtractive disentanglement framework. By semantically aligning modalities and subtracting the shared geometric context, it isolates intrinsic material features. Furthermore, GaMi incorporates inter-sample contrastive learning to correct the residual interference caused by cross-modal misalignment. Additionally, a pairing-based adaptation strategy between two modalities enables few-shot generalization across devices. Extensive evaluations on 20 materials show that GaMi achieves 95.2% accuracy, outperforming single-modality baselines across unseen geometric conditions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。