解析三层ReLU网络的对称性，揭示参数等价与梯度演化规律
The Symmetries of Three-Layer ReLU Networks
- 构建深度ReLU网络参数对称性分析框架，给出三层数学描述
- 提出多项式时间算法判断参数是否功能等价
- 发现部分对称性引发梯度流局部守恒，适于模型优化研究
我们建立了一个分析深层ReLU网络参数对称性的框架，完整刻画了三层瓶颈结构的典型参数纤维。该方法提供这些纤维的显式半代数描述，并给出一个多项式时间算法来判断两个参数是否功能等价。对称性包括由层间组合产生的离散与连续变换，其性质取决于深层是否隐藏或保留前层的几何结构。最后，我们证明其中一些对称性在梯度流中诱导局部守恒律，而另一些则不具有此性质。
原文摘要 · Abstract (English)
We develop a framework for analyzing parameter symmetries in deep ReLU networks and obtain a complete characterization of the generic parameter fibers for three-layer bottleneck architectures. Our approach provides explicit semi-algebraic descriptions of these fibers and yields a polynomial time algorithm for deciding functional equivalence of two parameters. The symmetries include discrete and continuous transformations arising from layer composition, and depend on whether deeper layers hide or preserve geometric structure from preceding layers. Finally, we show that some of these symmetries induce local conservation laws along gradient flow, while others do not.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。