动态调整核函数特征值，提升模型泛化能力。
Diagonal Over-parameterization in Reproducing Kernel Hilbert Spaces as an Adaptive Feature Model: Generalization and Adaptivity
- 通过训练同时学习核特征值与输出系数
- 初始核不匹配时泛化性能显著优于固定核方法
- 揭示深度增强自适应性的内在机制
本文提出一种对角自适应核模型，可在训练过程中同时动态学习核特征值和输出系数。不同于依赖神经正切核理论的固定核方法，该模型能自适应地匹配目标函数结构，尤其在初始核与真实函数不匹配时，显著提升泛化性能。进一步证明，这种自适应性源于训练中学习到正确的特征值，体现出特征学习行为。通过拓展至更深参数化，揭示了额外深度如何增强模型的适应性和泛化能力。本研究融合特征学习与隐式正则化思想，为超越核区间的神经网络自适应性与泛化潜力提供了新视角。
原文摘要 · Abstract (English)
This paper introduces a diagonal adaptive kernel model that dynamically learns kernel eigenvalues and output coefficients simultaneously during training. Unlike fixed-kernel methods tied to the neural tangent kernel theory, the diagonal adaptive kernel model adapts to the structure of the truth function, significantly improving generalization over fixed-kernel methods, especially when the initial kernel is misaligned with the target. Moreover, we show that the adaptivity comes from learning the right eigenvalues during training, showing a feature learning behavior. By extending to deeper parameterization, we further show how extra depth enhances adaptability and generalization. This study combines the insights from feature learning and implicit regularization and provides new perspective into the adaptivity and generalization potential of neural networks beyond the kernel regime.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。