揭示神经网络宽度与高斯过程极限的定量关系，适用于多种架构。
Quantitative Gaussian-Process limits of Tensor Programs
- 用张量程序框架分析无限宽神经网络的高斯过程极限
- 给出宽度为n时误差不超过1/√n的精确界
- 方法通用，适用于前馈、循环及Transformer结构
我们通过张量程序的视角研究随机神经网络在无限宽度下的高斯过程极限，并提供了 Wasserstein 距离下的定量收敛理论。主要结果给出了有限宽度下网络输出与高斯过程极限之间误差的显式上界，其阶为宽度平方根的倒数。该框架具有架构无关性，涵盖前馈模型以及适用于循环和 Transformer 类架构的权值共享机制。
原文摘要 · Abstract (English)
We study the infinite-width Gaussian-process limit of random neural networks through the lens of tensor programs, and we provide a quantitative convergence theory in Wasserstein distance. Our main result gives explicit finite-width error bounds, of order inverse square-root of the widths between finite-network executions and their Gaussian-process limits. The framework is architecture-agnostic and covers feed-forward models together with weight-sharing schemes relevant for recurrent and transformer-type architectures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。