密集连接的神经网络无法逼近所有连续函数，存在固有局限。
Neural Networks With Dense Weights Are Not Universal Approximators
- 将前馈网络视为消息传递图网络，结合弱正则性引理分析
- 在权重和维度受限下，存在无法逼近的Lipschitz连续函数
- 揭示密集结构的理论缺陷，支持稀疏连接更优
我们研究了密集神经网络的逼近能力。尽管通用逼近定理表明，在无权重限制时足够大的网络可逼近任意连续函数，但我们证明密集神经网络不具备这种普遍性。该结论基于模型压缩方法，结合弱正则性引理，并将前馈网络解释为消息传递图神经网络。在考虑输入输出维度及权重的自然约束条件下，我们证明存在某些Lipschitz连续函数无法被此类网络逼近。这揭示了带密集层的神经网络的内在局限性，强调稀疏连接是实现真正通用逼近的必要条件。
原文摘要 · Abstract (English)
We investigate the approximation capabilities of dense neural networks. While universal approximation theorems establish that sufficiently large architectures can approximate arbitrary continuous functions if there are no restrictions on the weight values, we show that dense neural networks do not possess this universality. Our argument is based on a model compression approach, combining the weak regularity lemma with an interpretation of feedforward networks as message passing graph neural networks. We consider ReLU neural networks subject to natural constraints on weights and input and output dimensions, which model a notion of dense connectivity. Within this setting, we demonstrate the existence of Lipschitz continuous functions that cannot be approximated by such networks. This highlights intrinsic limitations of neural networks with dense layers and motivates the use of sparse connectivity as a necessary ingredient for achieving true universality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。