新模型IGD可同时生成离散与连续数据,自动处理隐含约束。
Interleaved Gibbs Diffusion: Generating Discrete-Continuous Data with Implicit Constraints
- 用吉布斯采样思想设计非独立去噪过程,增强变量间依赖建模。
- 在3-SAT、分子结构等任务上表现领先,无需领域先验知识。
- 支持条件生成和推理优化,适合复杂混合数据生成场景。
我们提出一种新型生成模型Interleaved Gibbs Diffusion(IGD),用于建模包含重要隐含约束的离散-连续数据。现有离散扩散模型多假设去噪分布可分解,限制了变量间强依赖的捕捉能力。实验表明,改用非分解的吉布斯采样式离散扩散,即可在3-SAT任务上实现显著性能提升。受此启发,IGD将离散时间吉布斯采样推广至离散-连续生成场景,实现离散与连续去噪器的无缝融合,并理论保证正向过程的精确反演。该框架还支持条件生成(通过状态空间加倍)与推理时精炼。在分子结构、布局生成和表格数据三个挑战性任务上,IGD均达到当前最优性能,且不依赖如等变扩散或辅助损失等特定领域先验。我们系统探索了各类建模与交错策略及超参数配置。
原文摘要 · Abstract (English)
We introduce Interleaved Gibbs Diffusion (IGD), a novel generative modeling framework for discrete-continuous data, focusing on problems with important, implicit and unspecified constraints in the data. Most prior works on discrete and discrete-continuous diffusion assume a factorized denoising distribution, which can hinder the modeling of strong dependencies between random variables in such problems. We empirically demonstrate a significant improvement in 3-SAT performance out of the box by switching to a Gibbs-sampling style discrete diffusion model which does not assume factorizability. Motivated by this, we introduce IGD which generalizes discrete time Gibbs sampling type Markov chain for the case of discrete-continuous generation. IGD allows for seamless integration between discrete and continuous denoisers while theoretically guaranteeing exact reversal of a suitable forward process. Further, it provides flexibility in the choice of denoisers, allows conditional generation via state-space doubling and inference time refinement. Empirical evaluations on three challenging generation tasks - molecule structures, layouts and tabular data - demonstrate state-of-the-art performance. Notably, IGD achieves state-of-the-art results without relying on domain-specific inductive biases like equivariant diffusion or auxiliary losses. We explore a wide range of modeling, and interleaving strategies along with hyperparameters in each of these problems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。