提出可解的球形玻尔兹曼机，揭示能量模型训练与生成的本质机制。
Spherical Boltzmann machines: a solvable theory of learning and generation in energy-based models
- 基于随机矩阵与平均场理论，精确求解高维球形玻尔兹曼机的训练动态。
- 发现训练中存在相变级联，关联耦合矩阵模式对数据的逐级对齐与凝聚。
- 揭示采样温度调节、正则化双下降等现象，适用于主流生成模型。
能量基模型（EBM）是受统计物理启发的灵活生成架构，但其学习与生成特性仍不清晰。本文分析高维极限下的可解模型——球形玻尔兹曼机（SBM），结合随机矩阵理论与动力学平均场理论：求解了SBM训练动态的精确方程；计算了贝叶斯证据（作为参数空间的配分函数），编码训练后模型的全局性质；揭示了训练过程及超参数变化下的相变级联，与耦合矩阵最高模式对数据的逐级对齐和凝聚相关。这些相变与教师-学生场景中的采样生成现象相连，包括：采样温度调节、正则化强度下的双下降现象、温控后验效应，以及训练中非平衡效应导致的模型偏差。数值证据表明，这些现象同样存在于标准生成架构中，不限于SBM。
原文摘要 · Abstract (English)
Energy-based models (EBMs) are flexible generative architectures inspired by statistical physics, but their learning and generative properties remain poorly understood. Here, we analyze a solvable EBM in the high-dimensional limit: the spherical Boltzmann machine (SBM). Combining tools from random matrix theory and dynamical mean-field theory, we: solve exact equations describing the training dynamics of the SBM; compute the Bayesian evidence, which acts as a partition function in parameter space and encodes global properties of the trained model; and uncover cascades of phase transitions that occur both during training and as a function of hyperparameters, related to successive alignment and condensation of the top modes of the coupling matrix to the data. We connect these transitions to sampling-time generative phenomena in a teacher-student scenario, including: sampling temperature tuning, double descent as a function of regularization strength, tempered posterior effects, and out-of-equilibrium effects during training that induce biases in the trained model. We provide numerical evidence demonstrating that all these phenomena appear in standard generative architectures, beyond the SBM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。