将统计先验融入能量模型，提升生成效果与数据拟合能力。
Incorporating Inductive Biases to Energy-based Generative Models
- 用指数族模型补充能量函数，引入无参数统计特征增强先验
- 在近似最大化似然下仍能匹配数据关键统计量
- 适合需要强先验约束的生成任务，如图像/信号建模
随着基于得分匹配的训练方法和朗之万动力学采样技术的发展，能量模型（EBM）重新成为生成建模的热门选择。现有EBM通常使用神经网络定义能量函数。本文提出一种新型混合方法，将EBM与指数族模型结合，以在数据建模中引入归纳偏置。具体而言，在能量项中加入无参数统计函数，帮助模型捕捉数据的关键统计特性。与指数族模型类似，该混合模型在训练中旨在对齐分布统计量与数据统计量，即使仅近似最大化数据似然。这一性质使我们能够对模型施加约束。实证研究验证了该混合模型在统计匹配方面的有效性。实验结果表明，当引入合适的有信息量统计量时，模型的数据拟合与生成性能均得到提升。
原文摘要 · Abstract (English)
With the advent of score-matching techniques for model training and Langevin dynamics for sample generation, energy-based models (EBMs) have gained renewed interest as generative models. Recent EBMs usually use neural networks to define their energy functions. In this work, we introduce a novel hybrid approach that combines an EBM with an exponential family model to incorporate inductive bias into data modeling. Specifically, we augment the energy term with a parameter-free statistic function to help the model capture key data statistics. Like an exponential family model, the hybrid model aims to align the distribution statistics with data statistics during model training, even when it only approximately maximizes the data likelihood. This property enables us to impose constraints on the hybrid model. Our empirical study validates the hybrid model's ability to match statistics. Furthermore, experimental results show that data fitting and generation improve when suitable informative statistics are incorporated into the hybrid model.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。