发现高斯核与神经网络在无界域上存在本质差异，部分函数可被核方法表示却无法被神经网络捕捉。
A Gap Between the Gaussian RKHS and Neural Networks: An Infinite-Center Asymptotic Analysis
- 通过无限中心渐近分析，揭示无界域下神经网络的函数空间与高斯核空间的差异
- 证明某些高斯RKHS中的函数在神经网络范数下为无穷大，形成非平凡差距
- 适用于研究深度学习泛化性与核方法局限性的理论工作者
近期研究将无限宽、有界范数的单隐层神经网络的函数空间归纳偏置刻画为一类有变差类型的巴拿赫空间。该空间包含许多经典多元函数空间，如特定的索伯列夫空间和谱巴龙空间，也涵盖仅沿少数方向变化的低光滑性函数。在有界域上，高斯再生核希尔伯特空间（RKHS）严格嵌入该巴拿赫空间，表明二者存在明显差距。然而在无界域（如全空间ℝᵈ）上，情况截然不同：我们建立了基本结论——某些属于高斯RKHS的函数在神经网络巴拿赫空间中具有无穷范数，从而在理论上确立了核方法与神经网络之间的非平凡差距，即核方法可轻易表示的函数，神经网络却无法捕捉。
原文摘要 · Abstract (English)
Recent works have characterized the function-space inductive bias of infinite-width bounded-norm single-hidden-layer neural networks as a kind of bounded-variation-type space. This novel neural network Banach space encompasses many classical multivariate function spaces, including certain Sobolev spaces and the spectral Barron spaces. Notably, this Banach space also includes functions that exhibit less classical regularity, such as those that only vary in a few directions. On bounded domains, it is well-established that the Gaussian reproducing kernel Hilbert space (RKHS) strictly embeds into this Banach space, demonstrating a clear gap between the Gaussian RKHS and the neural network Banach space. It turns out that when investigating these spaces on unbounded domains, e.g., all of $\mathbb{R}^d$, the story is fundamentally different. We establish the following fundamental result: Certain functions that lie in the Gaussian RKHS have infinite norm in the neural network Banach space. This provides a nontrivial gap between kernel methods and neural networks by exhibiting functions that kernel methods easily represent, whereas neural networks cannot.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。