利用梯度低秩特性提升分布外检测可靠性
GradPCA: Leveraging NTK Alignment for Reliable Out-of-Distribution Detection
- 基于神经正切核对齐的梯度低秩结构设计检测方法
- 在多个图像分类基准上表现更稳定，优于现有方法
- 适合关注可靠检测与预训练特征质量的研究者
我们提出 GradPCA，一种利用神经正切核（NTK）对齐引发的神经网络梯度低秩结构的分布外（OOD）检测方法。该方法对梯度类别均值进行主成分分析（PCA），在标准图像分类基准上表现出比现有方法更一致的性能。我们从谱检测的角度提供了理论视角，揭示了有效检测所依赖的特征空间性质，并指出这些性质自然源于 NTK 对齐。分析进一步表明，特征质量——尤其是预训练与非预训练表示的差异——是决定检测器成败的关键因素。大量实验验证了 GradPCA 的优异表现，其理论框架也为设计更严谨的谱型 OOD 检测器提供了指导。
原文摘要 · Abstract (English)
We introduce GradPCA, an Out-of-Distribution (OOD) detection method that exploits the low-rank structure of neural network gradients induced by Neural Tangent Kernel (NTK) alignment. GradPCA applies Principal Component Analysis (PCA) to gradient class-means, achieving more consistent performance than existing methods across standard image classification benchmarks. We provide a theoretical perspective on spectral OOD detection in neural networks to support GradPCA, highlighting feature-space properties that enable effective detection and naturally emerge from NTK alignment. Our analysis further reveals that feature quality -- particularly the use of pretrained versus non-pretrained representations -- plays a crucial role in determining which detectors will succeed. Extensive experiments validate the strong performance of GradPCA, and our theoretical framework offers guidance for designing more principled spectral OOD detectors.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。