神经网络的泛化能力不依赖于特征学习质量。
Feature learning is decoupled from generalization in high capacity neural networks
- 提出'特征质量'概念衡量特征学习效果
- 实证发现现有理论只关注学习强度而非质量
- 为泛化理论研究提供新方向，适合理论研究者
神经网络在某些任务上（如阶梯函数）性能远超核方法，这得益于其特征学习能力——能够调整隐藏表示以更好捕捉数据。本文引入'特征质量'这一概念来量化这种性能提升。我们检验了现有的特征学习理论，发现它们主要评估特征学习的强度，而非所学特征的实际质量。因此，当前的特征学习理论无法为神经网络泛化理论提供充分基础。
原文摘要 · Abstract (English)
Neural networks outperform kernel methods, sometimes by orders of magnitude, e.g. on staircase functions. This advantage stems from the ability of neural networks to learn features, adapting their hidden representations to better capture the data. We introduce a concept we call feature quality to measure this performance improvement. We examine existing theories of feature learning and demonstrate empirically that they primarily assess the strength of feature learning, rather than the quality of the learned features themselves. Consequently, current theories of feature learning do not provide a sufficient foundation for developing theories of neural network generalization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。