arXiv:2412.01019stat.MLcs.LG2024-12NeurIPS被引 5

用热方程在离散空间建模,无需采样即可训练能量模型。

Energy-Based Modelling for Discrete and Mixed Data via Heat Equations on Structured Spaces

  • 基于图结构的扩散过程生成数据扰动,构建能量损失函数。
  • 在非二值词汇密度估计和二值图像建模中实现高精度生成。
  • 适合处理离散与混合数据,尤其适用于表格数据建模。

能量基模型(EBM)为多种数据领域的概率建模提供了灵活框架。然而,在离散或混合状态空间上训练EBM面临显著挑战,主要源于缺乏高效可靠的采样方法。本文提出一种基于能量差异(Energy Discrepancy)的损失函数,仅需评估数据点及其扰动样本的能量值,从而避免使用马尔可夫链蒙特卡洛采样。通过在具有图结构的离散状态空间上模拟扩散过程,生成数据扰动,使扰动选择能反映模型变量的结构特征,且连续时间参数可精细控制扰动程度。实验表明,该方法在广泛任务中表现优异,包括非二值词汇的离散密度估计、二值图像建模,以及在表格数据集上的合成数据生成与校准分类任务。

原文摘要 · Abstract (English)

Energy-based models (EBMs) offer a flexible framework for probabilistic modelling across various data domains. However, training EBMs on data in discrete or mixed state spaces poses significant challenges due to the lack of robust and fast sampling methods. In this work, we propose to train discrete EBMs with Energy Discrepancy, a loss function which only requires the evaluation of the energy function at data points and their perturbed counterparts, thus eliminating the need for Markov chain Monte Carlo. We introduce perturbations of the data distribution by simulating a diffusion process on the discrete state space endowed with a graph structure. This allows us to inform the choice of perturbation from the structure of the modelled discrete variable, while the continuous time parameter enables fine-grained control of the perturbation. Empirically, we demonstrate the efficacy of the proposed approaches in a wide range of applications, including the estimation of discrete densities with non-binary vocabulary and binary image modelling. Finally, we train EBMs on tabular data sets with applications in synthetic data generation and calibrated classification.

能量模型离散数据扩散过程表格数据

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。