通过调控神经坍缩，同时提升模型对异常数据的识别与迁移能力
Controlling Neural Collapse Enhances Out-of-Distribution Detection and Transfer Learning
- 分层控制神经坍缩程度，平衡检测与泛化性能
- 在多个数据集和模型上实现检测与迁移双优表现
- 适合需要高鲁棒性检测与良好迁移能力的场景
分布外（OOD）检测与泛化在深度神经网络中被广泛研究，但二者关系仍不明确。我们实证发现：网络层中神经坍缩（NC）程度与这两项任务呈反向关联——强NC利于检测但损害泛化，弱NC则相反。这表明单一特征空间难以兼顾二者。为此，我们建立理论框架，揭示NC与两者的关系：熵正则化可缓解NC以提升泛化，固定等角紧框架（ETF）投影器则强制NC以增强检测。基于此，我们提出一种在不同层调控NC的方法。实验表明，该方法在多种OOD数据集和网络架构下均显著优于基线。代码已公开于：https://yousuf907.github.io/ncoodg
原文摘要 · Abstract (English)
Out-of-distribution (OOD) detection and OOD generalization are widely studied in Deep Neural Networks (DNNs), yet their relationship remains poorly understood. We empirically show that the degree of Neural Collapse (NC) in a network layer is inversely related with these objectives: stronger NC improves OOD detection but degrades generalization, while weaker NC enhances generalization at the cost of detection. This trade-off suggests that a single feature space cannot simultaneously achieve both tasks. To address this, we develop a theoretical framework linking NC to OOD detection and generalization. We show that entropy regularization mitigates NC to improve generalization, while a fixed Simplex Equiangular Tight Frame (ETF) projector enforces NC for better detection. Based on these insights, we propose a method to control NC at different DNN layers. In experiments, our method excels at both tasks across OOD datasets and DNN architectures. Code for our experiments is available at: https://yousuf907.github.io/ncoodg
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。