深度学习的训练结果难以复现，根源在于其解空间具有无限精细的分形结构。
Riddled basin geometry sets fundamental limits to predictability and reproducibility in deep learning
- 解空间呈现分形且纠缠的几何结构，初始值微小变化可能引发结果剧变。
- 不确定性指数接近零，即使精度大幅提升，预测能力提升也极为有限。
- 解释了训练不一致、可复现性差等现象，适合关注AI可靠性的人参考。
可预测性的根本限制是理解物理与计算系统的核心问题。本文揭示，尽管深度学习表现卓越，其训练过程仍受制于分形且纠缠的吸引域几何结构：任何导致某一解的初始化，都与导致另一不同解的初始化任意邻近。通过将深度学习中广泛观察到的混沌训练动态与对称性诱导的不变子空间等特征联系起来，我们推导出分形吸引域出现的充分条件，揭示了真实深度网络中普遍存在的分形路径。由此产生的吸引域具有无限精细的分形结构，不确定性指数趋近于零，意味着即使大幅提高初始条件精度，结果可预测性的提升也微乎其微。分形结构因此构成了神经网络训练可预测性与可复现性的根本限制，统一解释了诸多经验现象。这些发现揭示了深度学习的一般组织原则，对优化与人工智能安全部署具有重要启示。
原文摘要 · Abstract (English)
Fundamental limits to predictability are central to our understanding of many physical and computational systems. Here we show that, despite its remarkable capabilities, deep learning exhibits such fundamental limits rooted in the fractal, riddled geometry of its basins of attraction: any initialization that leads to one solution lies arbitrarily close to another that leads to a different one. We derive sufficient conditions for the emergence of riddled basins by analytically linking features widely observed in deep learning, including chaotic learning dynamics and symmetry-induced invariant subspaces, to reveal a general route to riddling in realistic deep networks. The resulting basins of attraction possess an infinitely fine-scale fractal structure characterized by an uncertainty exponent near zero, so that even large increases in the precision of initial conditions yield only marginal gains in outcome predictability. Riddling thus imposes a fundamental limit on the predictability and hence reproducibility of neural network training, providing a unified account of many empirical observations. These results reveal a general organizing principle of deep learning with important implications for optimization and the safe deployment of artificial intelligence.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。