改进反向蒸馏,提升异常检测准确率
Unlocking the Potential of Reverse Distillation for Anomaly Detection
- 设计专家教师-学生网络,同步优化编码器与解码器
- 在多个基准上超越现有无监督异常检测方法
- 适合需要高精度异常检测的工业场景
知识蒸馏(KD)是无监督异常检测(AD)的有前景方法。然而,学生网络的过度泛化常导致教师与学生在异常区域的表征差异减弱,引发检测失败。当前广泛采用的反向蒸馏(RD)范式通过编码器作教师、解码器作学生构建不对称结构,但该设计未能确保教师编码器有效区分正常与异常特征,也未保证学生解码器生成无异常特征。此外,缺乏跳跃连接导致特征重建中细节丢失。为此,我们提出带专家的反向蒸馏(RD with Expert),引入新型专家-教师-学生网络,实现对教师编码器和学生解码器的联合蒸馏。新增的专家网络增强学生生成正常特征的能力,优化教师对正常与异常特征的区分能力,减少漏检。同时,设计引导信息注入机制,过滤并传递教师到学生的特征,提升细节重建效果,降低误报率。在多个基准数据集上的实验表明,本方法在RD范式下显著优于现有无监督异常检测方法,充分释放了反向蒸馏的潜力。
原文摘要 · Abstract (English)
Knowledge Distillation (KD) is a promising approach for unsupervised Anomaly Detection (AD). However, the student network's over-generalization often diminishes the crucial representation differences between teacher and student in anomalous regions, leading to detection failures. To addresses this problem, the widely accepted Reverse Distillation (RD) paradigm designs the asymmetry teacher and student, using an encoder as teacher and a decoder as student. Yet, the design of RD does not ensure that the teacher encoder effectively distinguishes between normal and abnormal features or that the student decoder generates anomaly-free features. Additionally, the absence of skip connections results in a loss of fine details during feature reconstruction. To address these issues, we propose RD with Expert, which introduces a novel Expert-Teacher-Student network for simultaneous distillation of both the teacher encoder and student decoder. The added expert network enhances the student's ability to generate normal features and optimizes the teacher's differentiation between normal and abnormal features, reducing missed detections. Additionally, Guided Information Injection is designed to filter and transfer features from teacher to student, improving detail reconstruction and minimizing false positives. Experiments on several benchmarks prove that our method outperforms existing unsupervised AD methods under RD paradigm, fully unlocking RD's potential.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。