用稀疏自编码器挖掘中间层特征，让模型能看懂自己何时出错。
Sparse Autoencoders for Interpretable Out-of-Distribution Detection
- 在中间层用稀疏自编码器学习可解释的特征表示
- 新检测分数在标准基准上达到顶尖性能
- 适合需要理解模型误判原因的场景
可靠检测分布外（OOD）样本对机器学习模型的安全部署至关重要。神经网络在面对偏离训练数据的输入时往往产生过度自信的预测，导致性能显著下降。现有许多方法仅关注最终输出层，忽略了中间层丰富的层级信息。本文提出一种新方法，利用稀疏自编码器（SAEs）从中间激活中学习可解释特征。我们发现，分布内（ID）与分布外（OOD）数据会激活不同的稀疏特征集合。基于测试样本与ID类均值激活间的余弦相似度，提出新的OOD评分。该后处理检测方法不仅在标准基准上达到当前最优性能，还提供了对分布偏移如何影响模型表示的可解释洞察。
原文摘要 · Abstract (English)
Reliable detection of out-of-distribution (OOD) samples is crucial for the safe deployment of machine learning models. Neural networks often produce overconfident predictions for inputs that deviate from their training data, leading to significant degradation in performance. While many OOD detection methods focus on the final output layer, they neglect the rich hierarchical information present in intermediate network layers. This paper introduces a novel approach that leverages sparse autoencoders (SAEs) to learn interpretable features from these intermediate activations. We find that in-distribution (ID) and OOD data activate distinct sets of these sparse features. We propose a new OOD score derived from the cosine similarity between the sparse feature activations of a test sample and the mean activations of ID classes. Our post-hoc detection method not only achieves state-of-the-art performance on standard OOD detection benchmarks, but yields interpretable insights into how distribution shift affects learned representations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。