用目标能量函数改进受限玻尔兹曼机的训练，解决传统方法的欠拟合与模式崩溃问题。
Ratio Divergence Learning Using Target Energy in Restricted Boltzmann Machines: Beyond Kullback--Leibler Divergence Learning
- 引入比率散度学习，结合正向与反向KL散度优势
- 在高维模型下显著提升能量拟合与模式覆盖能力
- 适合需要稳定训练和完整分布建模的研究者
我们提出一种用于离散能量模型的比率散度（RD）学习方法，该方法利用训练数据和一个可解析的目标能量函数。将该方法应用于受限玻尔兹曼机（RBMs），这类模型是满足离散分布通用近似定理的最小模型。RD学习结合了正向和反向Kullback-Leibler散度（KLD）学习的优点，有效解决了正向KLD导致的欠拟合和反向KLD引发的模式崩溃这一“著名”问题。由于正向与反向KLD之和似乎已足够融合两者优势，我们在数值实验中将其作为直接基线以评估效果。实验表明，RD学习在能量函数拟合、模式覆盖和学习稳定性方面均显著优于其他方法，且随着目标模型维度增加，性能差距愈发明显。
原文摘要 · Abstract (English)
We propose ratio divergence (RD) learning for discrete energy-based models, a method that utilizes both training data and a tractable target energy function. We apply RD learning to restricted Boltzmann machines (RBMs), which are a minimal model that satisfies the universal approximation theorem for discrete distributions. RD learning combines the strength of both forward and reverse Kullback-Leibler divergence (KLD) learning, effectively addressing the "notorious" issues of underfitting with the forward KLD and mode-collapse with the reverse KLD. Since the summation of forward and reverse KLD seems to be sufficient to combine the strength of both approaches, we include this learning method as a direct baseline in numerical experiments to evaluate its effectiveness. Numerical experiments demonstrate that RD learning significantly outperforms other learning methods in terms of energy function fitting, mode-covering, and learning stability across various discrete energy-based models. Moreover, the performance gaps between RD learning and the other learning methods become more pronounced as the dimensions of target models increase.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。