针对工业检测中模态缺失问题,提出鲁棒融合框架RADAR,提升模型泛化能力。
Robust Modality-incomplete Anomaly Detection: A Modality-instructive Framework with Benchmark
- 设计模态不完整指令与超网络自适应参数,增强模型对缺失模态的适应性。
- 构建双伪混合模块,利用模态组合差异抑制过拟合,显著提升性能。
- 首个系统研究模态缺失场景的基准数据集MIIAD Bench,适合工业质检应用。
多模态工业异常检测(MIAD)利用3D点云与2D RGB图像识别产品异常区域,在工业质量检测中至关重要。然而传统方法假设所有模态均成对存在,忽视了真实场景中模态缺失的普遍性。基于此,本文首次系统研究模态不完整的工业异常检测(MIIAD),并联合专家构建包含多种模态缺失情形的MIIAD Bench基准数据集,以模拟不完善的训练环境。实验发现,多数现有MIAD方法在该基准上表现显著下降。为此,本文提出两阶段鲁棒模态感知融合与检测框架RADAR:首先,引入模态不完整指令,结合超网络实现自适应参数学习,使多模态Transformer能稳健应对各类缺失场景;其次,设计双伪混合模块,突出不同模态组合的独特性,有效缓解过拟合问题。实验表明,RADAR在自建的MIIAD数据集上显著优于传统MIAD方法,验证了其实际应用价值。
原文摘要 · Abstract (English)
Multimodal Industrial Anomaly Detection (MIAD), which utilizes 3D point clouds and 2D RGB images to identify abnormal regions in products, plays a crucial role in industrial quality inspection. However, traditional MIAD settings assume that all 2D and 3D modalities are paired, ignoring the fact that multimodal data collected from the real world is often imperfect due to missing modalities. Additionally, models trained on modality-incomplete data are prone to overfitting. Therefore, MIAD models that demonstrate robustness against modality-incomplete data are highly desirable in practice. To address this, we introduce a pioneering study that comprehensively investigates Modality-Incomplete Industrial Anomaly Detection (MIIAD), and under the guidance of experts, we construct the MIIAD Bench with rich modality-missing settings to account for imperfect learning environments with incomplete multimodal information. As expected, we find that most existing MIAD methods perform poorly on the MIIAD Bench, leading to significant performance degradation. To tackle this challenge, we propose a novel two-stage Robust modAlity-aware fusing and Detecting framewoRk, abbreviated as RADAR. Specifically: i) We propose Modality-incomplete Instruction to guide the multimodal Transformer to robustly adapt to various modality-incomplete scenarios, and implement adaptive parameter learning based on HyperNetwork. ii) Then, we construct a Double-Pseudo Hybrid Module to highlight the uniqueness of modality combinations, mitigating overfitting issues and further enhancing the robustness of the MIIAD model. Our experimental results demonstrate that the proposed RADAR significantly outperforms traditional MIAD methods on our newly created MIIAD dataset, proving its practical application value.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。