提出非线性等变网络新框架，统一多种对称神经网络结构
Equivariant non-linear maps for neural networks on homogeneous spaces
- 基于群作用下的对称性约束，推导出非线性层的广义可转向性条件
- 证明了该构造具有通用性，能涵盖G-CNN、注意力机制等主流架构
- 为设计新型等变网络提供理论指导，适合对称建模研究者
本文提出一种在齐次空间上构建非线性等变神经网络层的新框架。针对Cohen等人关于齐次空间上$G$-CNN的线性等变理论，该工作将结论推广至非线性情形。我们推导出非线性等变层必须满足的广义可转向性约束，并证明了该构造的普遍性。对等变算子在特征图与群元素间对称性约束函数关系的深入理解，为未来等变网络设计提供了依据。我们展示了多种常见等变架构——$G$-CNN、隐式可转向核网络、常规及相对位置嵌入的注意力机制变压器、LieTransformers——均可由此框架统一导出。
原文摘要 · Abstract (English)
This paper presents a novel framework for non-linear equivariant neural network layers on homogeneous spaces. The seminal work of Cohen et al. on equivariant $G$-CNNs on homogeneous spaces characterized the representation theory of such layers in the linear setting, finding that they are given by convolutions with kernels satisfying so-called steerability constraints. Motivated by the empirical success of non-linear layers, such as self-attention or input dependent kernels, we set out to generalize these insights to the non-linear setting. We derive generalized steerability constraints that any such layer needs to satisfy and prove the universality of our construction. The insights gained into the symmetry-constrained functional dependence of equivariant operators on feature maps and group elements informs the design of future equivariant neural network layers. We demonstrate how several common equivariant network architectures - $G$-CNNs, implicit steerable kernel networks, conventional and relative position embedded attention based transformers, and LieTransformers - may be derived from our framework.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。