揭示归一化层在卷积网络中的平移等变性原理
Translation-Equivariance of Normalization Layers and Aliasing in Convolutional Neural Networks
- 建立归一化层对离散与连续平移等变性的理论框架
- 给出归一化层实现等变性的维度条件，经实证验证
- 适合关注物理可解释性与图像建模精度的研究者
设计严格满足连续平移等变性的卷积神经网络架构是当前研究热点，有助于提升科学计算中成像系统的物理准确性。现有工作多聚焦于下采样、上采样层和激活函数的设计，却较少关注归一化层。本文提出一种新的理论框架，用于理解归一化层对离散平移和连续平移的等变性，并确定了其在不同操作维度上实现等变性的充要条件。通过使用ResNet-18在ImageNet上的真实特征图进行实验，验证了理论预测的一致性。
原文摘要 · Abstract (English)
The design of convolutional neural architectures that are exactly equivariant to continuous translations is an active field of research. It promises to benefit scientific computing, notably by making existing imaging systems more physically accurate. Most efforts focus on the design of downsampling/pooling layers, upsampling layers and activation functions, but little attention is dedicated to normalization layers. In this work, we present a novel theoretical framework for understanding the equivariance of normalization layers to discrete shifts and continuous translations. We also determine necessary and sufficient conditions for normalization layers to be equivariant in terms of the dimensions they operate on. Using real feature maps from ResNet-18 and ImageNet, we test those theoretical results empirically and find that they are consistent with our predictions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。