利用中间层特征提升模型对分布外样本的识别能力
Leveraging Intermediate Representations for Better Out-of-Distribution Detection
- 通过能量对比损失正则化中间层特征
- 多层特征聚合后检测性能显著提升
- 适合需要高可靠性安全决策的场景
在真实应用中，机器学习模型必须可靠地检测分布外（OoD）样本，以避免产生危险决策。现有OoD检测方法通常依赖神经网络倒数第二层的输出或logits，但对中间层中蕴含的丰富信息利用不足。为此，我们分析了中间层的判别能力，证明其可有效用于OoD检测。提出一种基于能量的对比损失，对中间层进行正则化，并将多个中间层的响应聚合为统一表示。在多个数据集上进行的全面评估表明，该方法显著提升了OoD检测性能。
原文摘要 · Abstract (English)
In real-world applications, machine learning models must reliably detect Out-of-Distribution (OoD) samples to prevent unsafe decisions. Current OoD detection methods often rely on analyzing the logits or the embeddings of the penultimate layer of a neural network. However, little work has been conducted on the exploitation of the rich information encoded in intermediate layers. To address this, we analyze the discriminative power of intermediate layers and show that they can positively be used for OoD detection. Therefore, we propose to regularize intermediate layers with an energy-based contrastive loss, and by grouping multiple layers in a single aggregated response. We demonstrate that intermediate layer activations improves OoD detection performance by running a comprehensive evaluation across multiple datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。