提出可调控重采样机制,提升离散扩散模型生成质量
Interpolating Discrete Diffusion Models with Controllable Resampling

- 通过可控重采样减少对中间状态的依赖
- 在分子图与文本生成中达到领先性能
- 适合需要高质量离散生成的科研与工程场景
离散扩散模型在文本、图等多领域表现强大,但现有方法存在根本局限:掩码扩散模型因早期解码导致不可逆错误,均匀扩散模型虽支持自纠正却因过度依赖中间状态而生成质量低。本文提出IDDM——一种插值型离散扩散模型,通过可控重采样机制将部分概率质量重置至先验分布,缓解误差累积并实现更有效的标记修正。IDDM的生成过程在保持当前状态、从先验重采样和向目标状态翻转之间插值,同时保证边缘一致性且训练与推理完全解耦。在分子图生成和文本生成任务上,与最先进离散扩散模型对比,展现出竞争力。
原文摘要 · Abstract (English)
Discrete diffusion models form a powerful class of generative models across diverse domains, including text and graphs. However, existing approaches face fundamental limitations. Masked diffusion models suffer from irreversible errors due to early unmasking, while uniform diffusion models, despite enabling self-correction, often yield low-quality samples due to their strong reliance on intermediate latent states. We introduce IDDM, an Interpolating Discrete Diffusion Model, that improves diffusion by reducing dependence on intermediate latent states. Central to IDDM is a controllable resampling mechanism that partially resets probability mass to the marginal distribution, mitigating error accumulation and enabling more effective token corrections. IDDM specifies a generative process whose transitions interpolate between staying at the current state, resampling from a prior, and flipping toward the target state, while enforcing marginal consistency and fully decoupling training from inference. We benchmark our model against state-of-the-art discrete diffusion models across molecular graph generation as well as text generation tasks, demonstrating competitive performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。