深度等变网络的通用性有了新突破,揭示深度与读出层的关键作用。
On Universality of Deep Equivariant Networks
- 提出更严格的逐元素可分性概念,替代传统分离性
- 证明深层或带读出层的网络可逼近任意逐元素可分函数
- 统一并拓展了以往专用模型的通用性结论,适合理论研究者
等变神经网络的通用性结果仍极为稀少。现有结果通常局限于特定条件:要么依赖正则或高阶张量表示,导致隐藏空间维度过高而不实用;要么仅针对特殊架构,常局限于不变情形。本文提出更普适的分析框架。对于不变网络,在分离性约束下,我们证明添加全连接读出层即可逼近所有满足分离性约束的连续函数。对于等变网络,由于结果更为稀缺,我们指出传统可分性概念不足,引入更精细的“逐元素可分性”标准。证明在足够深度或加入适当读出层时,等变网络可在逐元素可分函数类中实现通用逼近。结合此前浅层模型不具通用性的结果,本工作明确深度与读出层是实现通用性的决定性机制,并提供统一视角,涵盖并扩展了早期特殊结果。
原文摘要 · Abstract (English)
Universality results for equivariant neural networks remain rare. Those that do exist typically hold only in restrictive settings: either they rely on regular or higher-order tensor representations, leading to impractically high-dimensional hidden spaces, or they target specialized architectures, often confined to the invariant setting. This work develops a more general account. For invariant networks, we establish a universality theorem under separation constraints, showing that the addition of a fully connected readout layer secures approximation within the class of separation-constrained continuous functions. For equivariant networks, where results are even scarcer, we demonstrate that standard separability notions are inadequate and introduce the sharper criterion of $\textit{entry-wise separability}$. We show that with sufficient depth or with the addition of appropriate readout layers, equivariant networks attain universality within the entry-wise separable regime. Together with prior results showing the failure of universality for shallow models, our findings identify depth and readout layers as a decisive mechanism for universality, additionally offering a unified perspective that subsumes and extends earlier specialized results.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。