神经网络能自动学习群结构的谱表示，揭示特征学习的数学机制。
Neural Networks Provably Learn Spectral Representations for Group Composition
- 通过傅里叶域梯度流分析，揭示训练动态由李群梯度上升主导。
- 单个神经元几乎必然收敛到单一不可约表示，跨层系数实现旋转秩一对齐。
- 适用于阿贝尔群的完整理论描述，支持快速相位对齐与表示竞争。
理解神经网络训练中结构化内部结构的涌现是深度学习研究的核心问题。我们通过群合成任务进行研究：用两层神经网络预测有限群 $G$ 中元素 $g_1 /star g_2$ 的结果。通过将投影梯度流提升至傅里叶域，我们证明训练动态遵循表示论能量泛函上的黎曼梯度上升。在随机初始化下，该流几乎必然驱动每个神经元收敛至单一不可约表示，且跨层傅里叶系数实现旋转秩一对齐。该框架提供了特征学习的表示论解释，并刻画了矩阵值群表示的一种新型低秩压缩现象。对于阿贝尔群，我们给出了完整的种群级描述：随机初始化促进非平凡表示的均匀分化，并诱导哈尔均匀相位，联合通过多数投票机制近似指示函数。我们进一步证明相位对齐与表示竞争均以指数速率出现。
原文摘要 · Abstract (English)
Understanding how structured internal structure emerges during neural network training is central to the study of deep learning. We investigate this phenomenon through the group composition task, where a two-layer neural network is trained to predict $g_1 \star g_2$ for elements of a finite group $G$. By lifting the projected gradient flow to the Fourier domain, we demonstrate that the training dynamics are governed by a Riemannian gradient ascent on a representation-theoretic energy functional. We prove that, under random initialization, this flow drives each neuron to converge almost surely toward a single irreducible representation, while the cross-layer Fourier coefficients achieve a rotational rank-one alignment. This framework provides a representation-theoretic account of feature learning and characterizes a novel low-rank compression phenomenon for matrix-valued group representations. Moreover, for Abelian groups, we provide a complete population-level description: random initialization promotes uniform diversification across nontrivial representations and induces Haar-uniform phases, jointly approximating the indicator via a majority-vote mechanism. We further prove that both phase alignment and representation competition emerge with exponential convergence rates.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。