提出特征误差度量，证明多模型训练中特征学习持续优化。
Supporting Evidence for the Adaptive Feature Program across Diverse Models
- 引入特征误差度量（FEM）量化特征学习质量
- 在多种模型中验证FEM随训练单调下降
- 为自适应特征程序的有效性提供理论支撑
理论上探索神经网络的优势是人工智能时代最具挑战性的问题之一。自适应特征程序被提出以更抽象的方式分析神经网络的特征学习特性。受经典Le Cam等价性的启发，我们主张使用过参数化序列模型来简化自适应特征程序的训练动态分析，并提供了若干支持性证据。具体而言，在引入特征误差度量（FEM）以表征所学特征的质量后，我们证明了在包括线性回归、单/多指数模型在内的多种具体自适应特征模型中，FEM在训练过程中持续下降。这一现象暗示了自适应特征程序具有潜在成功可能性。
原文摘要 · Abstract (English)
Theoretically exploring the advantages of neural networks might be one of the most challenging problems in the AI era. An adaptive feature program has recently been proposed to analyze feature learning, the characteristic property of neural networks, in a more abstract way. Motivated by the celebrated Le Cam equivalence, we advocate the over-parameterized sequence models to further simplify the analysis of the training dynamics of adaptive feature program and present several pieces of supporting evidence for the adaptive feature program. More precisely, after having introduced the feature error measure (FEM) to characterize the quality of the learned feature, we show that the FEM is decreasing during the training process of several concrete adaptive feature models including linear regression, single/multiple index models, etc. We believe that this hints at the potential successes of the adaptive feature program.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。