揭示深度特征提取器中信息传播速度的理论极限，指导高效模型设计
Digital implementations of deep feature extractors are intrinsically informative
- 建立统一框架，推导不同神经网络在欧氏与非欧空间的信息传播上限
- 证明离散输入特征提取器与基于LCA群的卷积网络均呈现全局指数能量衰减
- 提供结构化信号域信息以优化传播速率，适合模型优化与理论研究者
深度特征提取器中快速的信息（能量）传播对于平衡计算复杂度与表征能力至关重要。本文在涵盖不同神经网络模型、适用于欧氏与非欧域的统一框架下，证明了能量传播速度的上界。利用信号域的额外结构信息，可显式确定或改进能量衰减速率。为验证此理论，我们展示了两类情形下的全局指数能量衰减：1）具有离散域输入信号的特征提取器；2）通过局部紧阿贝尔（LCA）群上的散射实现的卷积神经网络（CNNs）。
原文摘要 · Abstract (English)
Rapid information (energy) propagation in deep feature extractors is crucial to balance computational complexity versus expressiveness as a representation of the input. We prove an upper bound for the speed of energy propagation in a unified framework that covers different neural network models, both over Euclidean and non-Euclidean domains. Additional structural information about the signal domain can be used to explicitly determine or improve the rate of decay. To illustrate this, we show global exponential energy decay for a range of 1) feature extractors with discrete-domain input signals, and 2) convolutional neural networks (CNNs) via scattering over locally compact abelian (LCA) groups.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。