用权重矩阵特征值尾指数检测标签噪声，准确率预测力远超传统方法。
Spectral Signatures of Data Quality: Eigenvalue Tail Index as a Diagnostic for Label Noise in Neural Networks
- 通过分析网络瓶颈层的特征值尾指数，诊断标签噪声水平。
- 在21种噪声水平下，尾指数预测测试准确率的留一法决定系数达0.984。
- 适用于检测真实数据中的标注错误，尤其适合质量监控场景。
我们研究神经网络权重矩阵的谱特性是否可预测测试准确率。在受控标签噪声变化下，网络瓶颈层特征值分布的尾指数α，以留一法决定系数0.984（21种噪声水平，每水平3个随机种子）精准预测测试准确率，远超所有基线方法（最佳传统指标：最优层的Frobenius范数，留一法R²=0.149）。该关系在三种架构（MLP、CNN、ResNet-18）和两个数据集（MNIST、CIFAR-10）上均成立。但在固定数据质量下改变超参数（180种配置，涉及宽度、深度、学习率、权重衰减），所有谱特征与传统度量均为弱预测因子（R² < 0.25），其中全局L₂范数（留一法R²=0.219）略优于尾指数（留一法R²=0.167）。因此，尾指数被定义为数据质量诊断工具：对标签污染和训练集退化具有强检测力，而非通用泛化预测器。在合成噪声上校准的噪声检测器成功识别出CIFAR-10N中的真实人类标注错误（9%噪声检出，误差仅3%）。我们确定信息处理瓶颈层是该信号的来源，并将其与带刺随机矩阵模型的BBP相变联系起来。同时报告一负面结果：水平间距比⟨r⟩对权重矩阵无信息量，因满足Wishart普适性。
原文摘要 · Abstract (English)
We investigate whether spectral properties of neural network weight matrices can predict test accuracy. Under controlled label noise variation, the tail index alpha of the eigenvalue distribution at the network's bottleneck layer predicts test accuracy with leave-one-out R^2 = 0.984 (21 noise levels, 3 seeds per level), far exceeding all baselines: the best conventional metric (Frobenius norm of the optimal layer) achieves LOO R^2 = 0.149. This relationship holds across three architectures (MLP, CNN, ResNet-18) and two datasets (MNIST, CIFAR-10). However, under hyperparameter variation at fixed data quality (180 configurations varying width, depth, learning rate, and weight decay), all spectral and conventional measures are weak predictors (R^2 < 0.25), with simple baselines (global L_2 norm, LOO R^2 = 0.219) slightly outperforming spectral measures (tail alpha, LOO R^2 = 0.167). We therefore frame the tail index as a data quality diagnostic: a powerful detector of label corruption and training set degradation, rather than a universal generalization predictor. A noise detector calibrated on synthetic noise successfully identifies real human annotation errors in CIFAR-10N (9% noise detected with 3% error). We identify the information-processing bottleneck layer as the locus of this signature and connect the observations to the BBP phase transition in spiked random matrix models. We also report a negative result: the level spacing ratio <r> is uninformative for weight matrices due to Wishart universality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。