无需记忆库的多模态异常检测方法,融合图像与点云信息提升稳定性。
Memoryless Multimodal Anomaly Detection via Student-Teacher Network and Signed Distance Learning
- 用轻量级师生网络从图像和点云中学习正常模式
- 通过差异和符号距离函数生成异常评分图,性能媲美顶尖方法
- 适合无内存限制的实时多模态异常检测场景
无监督异常检测是计算机视觉中的挑战性任务,尽管基于2D的方法已广泛研究,但基于RGB图像与3D点云的多模态异常检测仍需深入探索。现有方法多借鉴2D中常用的记忆库机制,需额外存储多模态特征。本文提出一种新型无记忆库方法MDSS,采用轻量级师生网络与符号距离函数,分别从RGB图像和3D点云中学习正常样本特征,并互补两种模态的异常信息。具体而言,师生网络通过动态损失函数训练,利用正常图像及由点云生成的掩码,通过学生与教师输出差异生成异常评分图;同时,符号距离函数从正常点云中学习点到表面的有向距离,据此生成异常评分图。最终对齐两模态评分图得到全局异常评分图。实验表明,MDSS在性能上可媲美当前最优的记忆库方法Shape-guided,且更稳定,优于其他基线方法。
原文摘要 · Abstract (English)
Unsupervised anomaly detection is a challenging computer vision task, in which 2D-based anomaly detection methods have been extensively studied. However, multimodal anomaly detection based on RGB images and 3D point clouds requires further investigation. The existing methods are mainly inspired by memory bank based methods commonly used in 2D-based anomaly detection, which may cost extra memory for storing mutimodal features. In present study, a novel memoryless method MDSS is proposed for multimodal anomaly detection, which employs a light-weighted student-teacher network and a signed distance function to learn from RGB images and 3D point clouds respectively, and complements the anomaly information from the two modalities. Specifically, a student-teacher network is trained with normal RGB images and masks generated from point clouds by a dynamic loss, and the anomaly score map could be obtained from the discrepancy between the output of student and teacher. Furthermore, the signed distance function learns from normal point clouds to predict the signed distances between points and surface, and the obtained signed distances are used to generate anomaly score map. Subsequently, the anomaly score maps are aligned to generate the final anomaly score map for detection. The experimental results indicate that MDSS is comparable but more stable than the SOTA memory bank based method Shape-guided, and furthermore performs better than other baseline methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。