发现深度神经网络中自发对称破缺可产生类戈德斯通模，实现稳定信息传播。
Spontaneous symmetry breaking and Goldstone modes for deep information propagation
- 利用连续对称性破缺构造类戈德斯通模式，无需残差连接等稳定结构
- 在前馈与循环网络中均提升信息传播稳定性，改善训练与长序列建模性能
- 为理解深层网络信息流动机制提供新视角，适合研究模型稳定性与架构设计者
在物理系统中，连续对称性自发破缺会引发称为戈德斯通模的激发态，使信息能长距离、长时间相干传播。本文研究内部层在连续对称性下保持等变的深度神经网络，发现其可能支持类似戈德斯通的自由度。通过分析与实证，我们证明这些自由度可在网络深度和循环迭代中实现相干信号传播，无需依赖残差连接或归一化等架构稳定器。在前馈网络中，该机制提升了各层可训练性与表征多样性；在循环设置中，它通过跨循环迭代传播信息，显著改善RNN与GRU在长序列建模任务上的表现。
原文摘要 · Abstract (English)
In physical systems, whenever a continuous symmetry is spontaneously broken, the system possesses excitations called Goldstone modes, which allow coherent information propagation over long distances and times. In this work, we study deep neural networks whose internal layers are equivariant under a continuous symmetry and may therefore support analogous Goldstone-like degrees of freedom. We demonstrate, both analytically and empirically, that these degrees of freedom enable coherent signal propagation across depth and recurrent iterations, providing a mechanism for stable information flow without relying on architectural stabilizers such as residual connections or normalization. In feedforward networks, this results in improved trainability and representational diversity across layers. In recurrent settings, we demonstrate the same mechanism is valuable for long-term memory by propagating information over recurrent iterations, thereby improving performance of RNNs and GRUs on long-sequence modeling tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。