用热力学势指导生成模型训练，揭示其通过打破遍历性实现稳定功能的机制。
Knowledge as a Breaking of Ergodicity
- 引入热力学势引导二元自由度上的生成模型训练。
- 训练后模型自由能出现多个极小值，对应多个非遍历性束缚态。
- 多模型并行可缓解学习与检索困难，适合复杂分布建模场景。
我们构建了一个热力学势，用于指导定义在二元自由度集合上的生成模型的训练。在降低描述复杂度以使其计算可行时，该势能会发展出多个极小值，这与生成模型自身自由能中多个极小值的出现相对应。使用 N 个二元自由度时，训练样本的多样性通常远低于全相空间大小 2^N。我们认为，未被代表的构型应视为一个由巨大能量间隙与训练集构型分离的高温相。因此，训练相当于在自由能表面采样一个由多个独立束缚态组成的库，每个束缚态都打破了遍历性。这种遍历性破坏阻止了向包含近连续状态的高温相逃逸，因而对正常功能至关重要。然而，它也可能导致对训练集中代表性不足模式的访问受限。同时，库内遍历性破坏使得学习和检索变得更加复杂。作为补救措施，可同时使用多个生成模型——最多每个自由能极小值一个。
原文摘要 · Abstract (English)
We construct a thermodynamic potential that can guide training of a generative model defined on a set of binary degrees of freedom. We argue that upon reduction in description, so as to make the generative model computationally-manageable, the potential develops multiple minima. This is mirrored by the emergence of multiple minima in the free energy proper of the generative model itself. The variety of training samples that employ N binary degrees of freedom is ordinarily much lower than the size 2^N of the full phase space. The non-represented configurations, we argue, should be thought of as comprising a high-temperature phase separated by an extensive energy gap from the configurations composing the training set. Thus, training amounts to sampling a free energy surface in the form of a library of distinct bound states, each of which breaks ergodicity. The ergodicity breaking prevents escape into the near continuum of states comprising the high-temperature phase; thus it is necessary for proper functionality. It may however have the side effect of limiting access to patterns that were underrepresented in the training set. At the same time, the ergodicity breaking within the library complicates both learning and retrieval. As a remedy, one may concurrently employ multiple generative models -- up to one model per free energy minimum.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。