arXiv:2502.13450cs.LGcs.AI2025-02

新模型IGD可同时生成离散与连续数据,自动处理隐含约束。

Interleaved Gibbs Diffusion: Generating Discrete-Continuous Data with Implicit Constraints

  • 用吉布斯采样思想设计非独立去噪过程,增强变量间依赖建模。
  • 在3-SAT、分子结构等任务上表现领先,无需领域先验知识。
  • 支持条件生成和推理优化,适合复杂混合数据生成场景。

我们提出一种新型生成模型Interleaved Gibbs Diffusion(IGD),用于建模包含重要隐含约束的离散-连续数据。现有离散扩散模型多假设去噪分布可分解,限制了变量间强依赖的捕捉能力。实验表明,改用非分解的吉布斯采样式离散扩散,即可在3-SAT任务上实现显著性能提升。受此启发,IGD将离散时间吉布斯采样推广至离散-连续生成场景,实现离散与连续去噪器的无缝融合,并理论保证正向过程的精确反演。该框架还支持条件生成(通过状态空间加倍)与推理时精炼。在分子结构、布局生成和表格数据三个挑战性任务上,IGD均达到当前最优性能,且不依赖如等变扩散或辅助损失等特定领域先验。我们系统探索了各类建模与交错策略及超参数配置。

原文摘要 · Abstract (English)

We introduce Interleaved Gibbs Diffusion (IGD), a novel generative modeling framework for discrete-continuous data, focusing on problems with important, implicit and unspecified constraints in the data. Most prior works on discrete and discrete-continuous diffusion assume a factorized denoising distribution, which can hinder the modeling of strong dependencies between random variables in such problems. We empirically demonstrate a significant improvement in 3-SAT performance out of the box by switching to a Gibbs-sampling style discrete diffusion model which does not assume factorizability. Motivated by this, we introduce IGD which generalizes discrete time Gibbs sampling type Markov chain for the case of discrete-continuous generation. IGD allows for seamless integration between discrete and continuous denoisers while theoretically guaranteeing exact reversal of a suitable forward process. Further, it provides flexibility in the choice of denoisers, allows conditional generation via state-space doubling and inference time refinement. Empirical evaluations on three challenging generation tasks - molecule structures, layouts and tabular data - demonstrate state-of-the-art performance. Notably, IGD achieves state-of-the-art results without relying on domain-specific inductive biases like equivariant diffusion or auxiliary losses. We explore a wide range of modeling, and interleaving strategies along with hyperparameters in each of these problems.

生成模型扩散模型混合数据约束生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。