用网络碎片化衡量模型复杂度，能有效预测深度学习泛化性能。
Is network fragmentation a useful complexity measure?
- 基于输入与隐藏层的碎片化程度构建复杂度指标
- 在PGDL基准上实现良好泛化预测性能
- 适合研究模型泛化机制的研究者参考
深度神经网络分类器的输入空间常表现出‘碎片化’现象，即模型输出随输入微小变化而快速切换类别。这种碎片化程度在插值区达到峰值，符合双下降曲线趋势。本文在图像分类任务中研究该现象是否可预测泛化性能，提出基于碎片化的复杂度度量，并在PGDL（Predicting Generalization in Deep Learning）基准上取得良好表现。此外，我们发现：(i) 碎片化不仅存在于输入空间，也出现在隐藏表示中；(ii) 碎片化趋势与验证误差变化一致；(iii) 碎片化并非由权重范数增加直接导致。这些结果表明，碎片化是值得深入研究的泛化能力相关现象。
原文摘要 · Abstract (English)
It has been observed that the input space of deep neural network classifiers can exhibit `fragmentation', where the model function rapidly changes class as the input space is traversed. The severity of this fragmentation tends to follow the double descent curve, achieving a maximum at the interpolation regime. We study this phenomenon in the context of image classification and ask whether fragmentation could be predictive of generalization performance. Using a fragmentation-based complexity measure, we show this to be possible by achieving good performance on the PGDL (Predicting Generalization in Deep Learning) benchmark. In addition, we report on new observations related to fragmentation, namely (i) fragmentation is not limited to the input space but occurs in the hidden representations as well, (ii) fragmentation follows the trends in the validation error throughout training, and (iii) fragmentation is not a direct result of increased weight norms. Together, this indicates that fragmentation is a phenomenon worth investigating further when studying the generalization ability of deep neural networks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。