通过分析神经网络内部特征演化，发现隐式增强机制支持泛化性能。
Feature Dynamics as Implicit Data Augmentation: A Depth-Decomposed View on Deep Neural Network Generalization
- 观察浅层早期特征与深层后期特征的组合稳定性
- 该稳定性在未见和损坏数据上仍成立，但随机标签下失效
- 适合关注模型泛化机理的研究者阅读
为何深度网络具有良好的泛化能力？不同于经典泛化理论，我们不仅考察输入输出，还关注内部特征的演化过程。研究发现一种时间一致性现象：当早期检查点的浅层特征与后期检查点的深层特征结合时，预测保持稳定。这种稳定性并非简单的收敛结果，而是一种隐式的、结构化的数据增强，有助于泛化。我们证明该一致性在未见数据和含噪声数据上依然存在，但在语义结构被破坏（如标签随机化）时崩溃。统计检验进一步显示，SGD注入了沿少数主方向的各向异性噪声，强化其作为结构化变异源的作用。这些发现提出一个将特征动态与泛化相联系的概念视角，为未来设计测量特征演化演变的实用代理方法指明方向。
原文摘要 · Abstract (English)
Why do deep networks generalize well? In contrast to classical generalization theory, we approach this fundamental question by examining not only inputs and outputs, but the evolution of internal features. Our study suggests a phenomenon of temporal consistency that predictions remain stable when shallow features from earlier checkpoints combine with deeper features from later ones. This stability is not a trivial convergence artifact. It acts as a form of implicit, structured augmentation that supports generalization. We show that temporal consistency extends to unseen and corrupted data, but collapses when semantic structure is destroyed (e.g., random labels). Statistical tests further reveal that SGD injects anisotropic noise aligned with a few principal directions, reinforcing its role as a source of structured variability. Together, these findings suggest a conceptual perspective that links feature dynamics to generalization, pointing toward future work on practical surrogates for measuring temporal feature evolution.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。