深度网络中维度压缩让异常样本更易被识别，关键机制是信息丢失。
Dimensionality-induced information loss of outliers in deep neural networks
- 通过分析各层特征，发现维度降低是异常检测的关键机制。
- 低维权重导致异常样本信息大量丢失，引发误分类。
- 提出基于维度感知的检测方法，效果好且计算成本低。
分布外（OOD）检测是保障深度神经网络系统稳定可靠运行的关键问题。尽管已有诸多检测方法，但各处理步骤如何影响分布内（ID）与分布外（OOD）样本差异仍不明确。本文从多角度实验研究了特征表示的层依赖性，发现深度网络的内在低维化是理解特征向深层传播时OOD样本逐渐与ID样本分离的核心原因。基于此，我们提出了一个统一解释各类OOD样本特性的简单模型：低维权重会消除多数OOD样本的信息，导致对数据集偏见过度关注而引发误分类。此外，我们提出一种基于特征与权重对齐的维度感知型OOD检测方法，该方法在多种数据集上均表现优异，且计算成本更低。
原文摘要 · Abstract (English)
Out-of-distribution (OOD) detection is a critical issue for the stable and reliable operation of systems using a deep neural network (DNN). Although many OOD detection methods have been proposed, it remains unclear how the differences between in-distribution (ID) and OOD samples are generated by each processing step inside DNNs. We experimentally clarify this issue by investigating the layer dependence of feature representations from multiple perspectives. We find that intrinsic low dimensionalization of DNNs is essential for understanding how OOD samples become more distinct from ID samples as features propagate to deeper layers. Based on these observations, we provide a simple picture that consistently explains various properties of OOD samples. Specifically, low-dimensional weights eliminate most information from OOD samples, resulting in misclassifications due to excessive attention to dataset bias. In addition, we demonstrate the utility of dimensionality by proposing a dimensionality-aware OOD detection method based on alignment of features and weights, which consistently achieves high performance for various datasets with lower computational cost.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。