统一融合2D与3D信息,提升工业缺陷检测精度
BridgeNet: A Unified Multimodal Framework for Bridging 2D and 3D Industrial Anomaly Detection
- 分离深度与外观特征,支持跨模态异常生成
- 在MVTec-3D和Eyecandies数据集上超越当前最优方法
- 共享参数设计,实现2D/3D检测无缝衔接
针对工业缺陷检测中仅依赖2D图像难以捕捉3D深度异常的问题,本文提出统一的多模态框架BridgeNet。通过从3D点云中提取可见深度信息,并用2D RGB图像表征外观,实现深度与外观的解耦,便于统一生成异常样本。引入多尺度高斯异常生成器和统一纹理异常生成器,在RGB与深度通道中生成更丰富的异常。所有模块共享参数,使2D与3D特征可直接融合,无需复杂融合操作。实验表明,该方法在MVTec-3D AD和Eyecandies数据集上均优于现有最先进方法。代码已开源。
原文摘要 · Abstract (English)
Industrial anomaly detection for 2D objects has gained significant attention and achieved progress in anomaly detection (AD) methods. However, identifying 3D depth anomalies using only 2D information is insufficient. Despite explicitly fusing depth information into RGB images or using point cloud backbone networks to extract depth features, both approaches struggle to adequately represent 3D information in multimodal scenarios due to the disparities among different modal information. Additionally, due to the scarcity of abnormal samples in industrial data, especially in multimodal scenarios, it is necessary to perform anomaly generation to simulate real-world abnormal samples. Therefore, we propose a novel unified multimodal anomaly detection framework to address these issues. Our contributions consist of 3 key aspects. (1) We extract visible depth information from 3D point cloud data simply and use 2D RGB images to represent appearance, which disentangles depth and appearance to support unified anomaly generation. (2) Benefiting from the flexible input representation, the proposed Multi-Scale Gaussian Anomaly Generator and Unified Texture Anomaly Generator can generate richer anomalies in RGB and depth. (3) All modules share parameters for both RGB and depth data, effectively bridging 2D and 3D anomaly detection. Subsequent modules can directly leverage features from both modalities without complex fusion. Experiments show our method outperforms state-of-the-art (SOTA) on MVTec-3D AD and Eyecandies datasets. Code available at: https://github.com/Xantastic/BridgeNet
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。