揭示神经网络参数空间对称性如何影响模型训练与泛化
Symmetry in Neural Network Parameter Spaces
- 分析参数空间中的对称变换如何让不同参数产生相同输出
- 发现对称性塑造损失曲面，影响优化路径和泛化能力
- 适合研究深度学习理论、优化算法的学者参考
现代深度学习模型高度过参数化，导致存在大量能产生相同输出的参数配置。其中相当一部分冗余源于参数空间中的对称性——即不改变网络函数的变换。这些对称性影响损失曲面结构，限制学习动态，为理解优化、泛化与模型复杂性提供了新视角，补充了现有深度学习理论。本文综述参数空间对称性的研究现状，梳理已有文献，揭示其与学习理论的关联，并指出该新兴领域中的研究空白与机遇。
原文摘要 · Abstract (English)
Modern deep learning models are highly overparameterized, resulting in large sets of parameter configurations that yield the same outputs. A significant portion of this redundancy is explained by symmetries in the parameter space--transformations that leave the network function unchanged. These symmetries shape the loss landscape and constrain learning dynamics, offering a new lens for understanding optimization, generalization, and model complexity that complements existing theory of deep learning. This survey provides an overview of parameter space symmetry. We summarize existing literature, uncover connections between symmetry and learning theory, and identify gaps and opportunities in this emerging field.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。