用统计物理讲透神经网络背后的能量原理与学习机制
Lecture Notes on Statistical Physics and Neural Networks
- 从自旋玻璃模型出发，用能量函数统一解释神经网络架构
- 揭示受限玻尔兹曼机学习中隐层积分与重整化群的相似性
- 适合对深度学习底层原理感兴趣的数学/物理背景读者
这些讲义介绍了经典统计物理的一些主题，特别是与神经网络和深度学习相关的内容。统计物理被视作概率论或统计学的一个分支，旨在让无物理背景的读者也能理解相变和重整化群等概念。我们引入了有限配置空间上的玻尔兹曼-吉布斯分布和热力学势，重点讨论了格点上的伊辛自旋和自旋玻璃模型，并将相变定义为当格点数趋于无穷时出现的不连续现象。接着介绍了霍普菲尔德网络和玻尔兹曼机，它们遵循与自旋玻璃模型相同的能量函数，并讨论了受限玻尔兹曼机的学习算法。该算法中隐层神经元的积分方式类似于重整化群。最后介绍了现代深度学习，其早期发展部分受到受限玻尔兹曼机的启发，具有多层隐层神经元结构。还简要描述了大规模语言模型。
原文摘要 · Abstract (English)
These lecture notes introduce some topics of classical statistical physics, particularly those that are relevant for neural networks and deep learning. Statistical physics is treated as a branch of probability theory or statistics, with the goal of making concepts such as phase transitions and the renormalization group accessible to readers without prior knowledge of physics. We introduce the Boltzmann-Gibbs distribution and the thermodynamic potentials on a finite configuration space, notably for Ising spins and spin-glass models on a lattice, and then define phase transitions as discontinuities that arise in the limit that the number of lattice points goes to infinity. We further introduce Hopfield networks and Boltzmann machines, which are governed by the same energy function as spin-glass models, and discuss the learning algorithm for restricted Boltzmann machines. In this algorithm hidden neurons are integrated out as in the renormalization group. Finally, modern deep learning is introduced, whose early developments were in part motivated by restricted Boltzmann machines in that they carry many layers of hidden neurons. A description of large language models is given.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。