提出通用谱正则化随机特征方法，理论分析神经网络学习率。
Random feature approximation for general spectral methods
- 基于谱正则框架统一分析梯度下降等优化算法的随机特征方法
- 在非再生核希尔伯特空间上实现最优学习率，覆盖更广函数类
- 为神经网络与神经算子提供理论支持，适合研究深度学习泛化者
随机特征近似是大规模学习算法中核方法最广泛应用的技术之一。本文分析了随机特征方法的泛化性质，将先前针对Tikhonov正则化的结果推广到一类广泛的谱正则化技术，涵盖显式方法及隐式算法（如梯度下降、Heavy-Ball和Nesterov加速方法）。通过该框架，我们得以从神经切线核（NTK）视角对梯度下降训练的神经网络和神经算子进行理论分析。对于所提出的估计器，在通过适当源条件定义的正则性类上实现了最优学习率，即使这些类不包含于再生核希尔伯特空间中。该结果改进或补全了以往特定核算法相关设置中的结论。
原文摘要 · Abstract (English)
Random feature approximation is arguably one of the most widely used techniques for kernel methods in large-scale learning algorithms. In this work, we analyze the generalization properties of random feature methods, extending previous results for Tikhonov regularization to a broad class of spectral regularization techniques. This includes not only explicit methods but also implicit schemes such as gradient descent and accelerated algorithms like the Heavy-Ball and Nesterov method. Through this framework, we enable a theoretical analysis of neural networks and neural operators through the lens of the Neural Tangent Kernel (NTK) approach trained via gradient descent. For our estimators we obtain optimal learning rates over regularity classes (even for classes that are not included in the reproducing kernel Hilbert space), which are defined through appropriate source conditions. This improves or completes previous results obtained in related settings for specific kernel algorithms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。