研究先验对受限玻尔兹曼机学习效率的影响，发现数据量临界点由教师模型决定。
The effect of priors on Learning with Restricted Boltzmann Machines
- 通过师生框架分析不同先验对学习的影响，先验介于高斯与二值之间。
- 发现存在三相点，决定泛化所需的最小数据量，该量受教师模型影响。
- 学生模型的先验选择可扩大有效泛化区域，利于训练但不影响临界数据量。
受限玻尔兹曼机（RBMs）是用于从具有复杂结构的数据中学习的生成模型。本文在师生设置下研究学生RBMs从教师RBMs生成的数据中学习的过程，重点分析单元先验对学习效率的影响。考虑一个参数化的先验类，其在连续（高斯）与二值变量之间插值，涵盖教师与学生模型中可见单元、隐藏单元及权重的各种可能配置。通过分析贝叶斯最优与不匹配情形下的后验分布相图，我们证明了三相点的存在，该点定义了通过泛化实现学习所需的关键数据集大小。这一临界大小强烈依赖于教师模型的特性，从而取决于数据本身，但不受学生模型属性影响。然而，合理选择学生先验可扩展所谓的信号检索区域，使机器更有效地泛化。
原文摘要 · Abstract (English)
Restricted Boltzmann Machines (RBMs) are generative models designed to learn from data with a rich underlying structure. In this work, we explore a teacher-student setting where a student RBM learns from examples generated by a teacher RBM, with a focus on the effect of the unit priors on learning efficiency. We consider a parametric class of priors that interpolate between continuous (Gaussian) and binary variables. This approach models various possible choices of visible units, hidden units, and weights for both the teacher and student RBMs. By analyzing the phase diagram of the posterior distribution in both the Bayes optimal and mismatched regimes, we demonstrate the existence of a triple point that defines the critical dataset size necessary for learning through generalization. The critical size is strongly influenced by the properties of the teacher, and thus the data, but is unaffected by the properties of the student RBM. Nevertheless, a prudent choice of student priors can facilitate training by expanding the so-called signal retrieval region, where the machine generalizes effectively.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。