提出可控制压缩率的卷积层激活降维方法,提升异常检测效率与性能。
A Convolutional Layer Activation Dimensionality Reduction for Out-of-Distribution and Adversarial Attack Detection Methods

- 设计可控压缩率的降维算法,平衡信息保留与维度降低。
- 在多种异常检测任务中表现优于或相当最强现有方法。
- 显著降低计算与内存开销,适合部署于资源受限场景。
尽管卷积神经网络在图像分类中取得成功,并广泛应用于多模态模型,但其对分布外和对抗攻击样本的敏感性引发可信度与安全性的担忧。针对此类问题,通过分析模型中间激活并估计置信度的检测方法具有前景,依赖降维步骤以高效处理高维激活。然而,现有卷积层降维方法要么缺乏压缩与信息损失的控制机制,要么产生较大表示。本文深入分析两种前沿检测方法及其卷积层降维策略,提出一种新型降维方法,支持可控高比例压缩。我们扩展了这两种检测方法,使其可兼容任意降维方式,并在分布外与对抗攻击检测上进行评估。结果表明,采用所提降维方法的检测系统在各项任务中均优于或媲美最强替代方案。此外,该方法在所有对比方法中压缩率最高,显著降低计算与内存开销。
原文摘要 · Abstract (English)
Despite the success of convolutional neural networks in image classification tasks and their general application in multi-modal models, their susceptibility to out-of-distribution and adversarial attack samples raises concerns regarding trustworthiness and safety. Among the approaches to tackle such issues, detection methods that analyze the model's intermediate activations to estimate a confidence score are a promising family that evaluates the decision process, relying on a dimensionality reduction step to enable efficient downstream processing of the high-dimensional activations. However, when considering convolutional layers, the dimensionality reduction methods in the literature either lack a mechanism to control the compression/information-loss trade-off or yield large representations. In this paper, we carefully analyze two state-of-the-art detection methods and their dimensionality reductions for convolutional layers and develop a novel reduction method with a controllable high-compression level. We extend these two state-of-the-art detection methods, enabling the usage of any dimensionality reduction, and evaluate their performance on out-of-distribution and adversarial attack detection. Results show that the detection methods with the proposed dimensionality reduction consistently perform better than, or comparable to, the strongest alternative. Furthermore, the proposed method is shown to reduce computation and memory footprints, given that it has the highest compression among the compared methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。