用微分几何方法为线性深度网络建立热力学模型，定义并计算其玻尔兹曼熵。
An entropy formula for the Deep Linear Network
- 通过群作用和李群轨道分解参数空间，构建可计算熵的几何结构。
- 证明可观测空间的黎曼度量由参数空间的子浸入映射而来。
- 适用于研究深度学习优化过程的几何与热力学性质的学者。
我们研究深度线性网络（DLN）的黎曼几何，作为学习过程热力学描述的基础。主要工具包括利用群作用分析过参数化，以及从参数空间到可观测空间的黎曼子浸入。参数空间中平衡流形的群轨道叶状结构被用来定义并计算玻尔兹曼熵。我们还证明，文献[2]中定义的可观测空间的黎曼几何，正是由平衡流形的黎曼子浸入所诱导的。关键技术步骤是利用雅可比矩阵理论，显式构造了平衡流形切空间的正交基。
原文摘要 · Abstract (English)
We study the Riemannian geometry of the Deep Linear Network (DLN) as a foundation for a thermodynamic description of the learning process. The main tools are the use of group actions to analyze overparametrization and the use of Riemannian submersion from the space of parameters to the space of observables. The foliation of the balanced manifold in the parameter space by group orbits is used to define and compute a Boltzmann entropy. We also show that the Riemannian geometry on the space of observables defined in [2] is obtained by Riemannian submersion of the balanced manifold. The main technical step is an explicit construction of an orthonormal basis for the tangent space of the balanced manifold using the theory of Jacobi matrices.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。