模型越复杂，越难识别异常数据？研究发现复杂度与检测性能存在双下降现象。
Double Descent Meets Out-of-Distribution Detection: Theoretical Insights and Empirical Analysis on the role of model complexity
- 通过后处理方式检测异常样本，研究模型复杂度对效果的影响
- 实证发现模型复杂度与检测性能呈双下降曲线，非越复杂越好
- 提出方法可定位最优复杂度区间，适合关注模型可靠性的人群
异常分布（OOD）检测对保障机器学习系统的可靠性与安全性至关重要。近年来，后处理检测和基于训练的方法受到广泛关注。本文聚焦于后处理型OOD检测，该方法无需修改模型训练过程或目标函数即可识别异常样本。核心目标是探究模型容量与检测性能之间的关系，具体回答：双下降现象是否在后处理型OOD检测中出现？这一问题至关重要，因为若过参数化能提升泛化能力，是否也能增强异常检测？尽管经典监督学习领域对此兴趣浓厚，但该交叉方向在OOD检测中仍属空白。我们通过实验证明，双下降现象确实在后处理型OOD检测中显现，并提供理论解释其成因。此外，我们发现过参数化并非始终表现更优，因此提出一种基于观察结果的最优复杂度识别方法。
原文摘要 · Abstract (English)
Out-of-distribution (OOD) detection is essential for ensuring the reliability and safety of machine learning systems. In recent years, it has received increasing attention, particularly through post-hoc detection and training-based methods. In this paper, we focus on post-hoc OOD detection, which enables identifying OOD samples without altering the model's training procedure or objective. Our primary goal is to investigate the relationship between model capacity and its OOD detection performance. Specifically, we aim to answer the following question: Does the Double Descent phenomenon manifest in post-hoc OOD detection? This question is crucial, as it can reveal whether overparameterization, which is already known to benefit generalization, can also enhance OOD detection. Despite the growing interest in these topics by the classic supervised machine learning community, this intersection remains unexplored for OOD detection. We empirically demonstrate that the Double Descent effect does indeed appear in post-hoc OOD detection. Furthermore, we provide theoretical insights to explain why this phenomenon emerges in such setting. Finally, we show that the overparameterized regime does not yield superior results consistently, and we propose a method to identify the optimal regime for OOD detection based on our observations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。