arXiv:2412.08949cs.CV2024-12中稿 · TMM

提出跨模态调优的反向蒸馏方法,提升工业多模态异常检测精度。

Tuned Reverse Distillation: Enhancing Multimodal Industrial Anomaly Detection with Crossmodal Tuners

  • 多分支设计分离各模态,独立捕捉单模态异常。
  • 引入跨模态滤波与增强器,提升模态间信息交互。
  • 在多个数据集上达到当前最优性能,适合工业质检场景。

知识蒸馏(KD)在无监督图像异常检测中广泛应用,但其在无监督多模态异常检测中的应用仍不充分。现有基于KD的多模态异常检测方法通过融合多模态特征生成教师模型,但仅存在于某一模态的异常可能无法有效体现在融合后的教师特征中,导致漏检。同时,这些方法未能充分利用模态内与模态间的关键信息。本文提出基于多分支设计的调优反向蒸馏(TRD),为每种模态分配独立分支,实现更精细的模态内异常检测。进一步,设计两种跨模态调优器(Crossmodal Filter和Amplifier),通过跨模态映射机制增强蒸馏过程中的模态交互,使学生网络更好地学习正常特征,同时确保所有模态的异常均被有效检测。在多个多模态异常检测数据集上的实验表明,该方法在异常检测与定位任务中达到当前最优性能。代码已开源:https://github.com/hito2448/TRD。

原文摘要 · Abstract (English)

Knowledge distillation (KD) has been widely studied in unsupervised image Anomaly Detection (AD), but its application to unsupervised multimodal AD remains underexplored. Existing KD-based methods for multimodal AD that use fused multimodal features to obtain teacher representations face challenges. Anomalies that only exist in one modality may not be effectively captured in the fused teacher features, leading to detection failures. Besides, these methods do not fully leverage the rich intra- and inter-modality information that are critical for effective anomaly detection. In this paper, we propose Tuned Reverse Distillation (TRD) based on Multi-branch design to realize Multimodal Industrial AD. By assigning independent branches to each modality, our method enables finer detection of anomalies within each modality. Furthermore, we enhance the interaction between modalities during the distillation process by designing two Crossmodal Tuners including Crossmodal Filter and Amplifier. With the idea of crossmodal mapping, the student network is allowed to better learn normal features while anomalies in all modalities are ensured to be effectively detected. Experimental verifications on multimodal AD datasets demonstrate that our method achieves state-of-the-art performance in multimodal anomaly detection and localization. Code is available at https://github.com/hito2448/TRD.

多模态异常检测蒸馏工业质检

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。