提出新方法让离散扩散模型更精准地生成目标内容。
Debiasing Guidance for Discrete Diffusion with Sequential Monte Carlo
- 用序贯蒙特卡洛算法实现无偏采样
- 文本生成时控制力强且困惑度低
- 适用于图像与文本的精准生成
离散扩散模型是在离散状态空间中逼近数据分布的生成模型。通常需要针对数据分布的特定区域进行采样。现有引导方法虽旨在从与 $p_0(x_0) p(ζ|x_0)^α$ 成比例的分布中采样，但实际难以实现。本文提出一种基于序贯蒙特卡洛的算法，利用学习到的无条件与引导过程，可无偏地从该目标分布中生成样本。我们在低维分布、受控图像和文本生成任务上验证了该方法的有效性。在文本生成中，该方法在保持低困惑度的同时展现出更强的控制能力，优于传统基于引导的方法。
原文摘要 · Abstract (English)
Discrete diffusion models are a class of generative models that produce samples from an approximated data distribution within a discrete state space. Often, there is a need to target specific regions of the data distribution. Current guidance methods aim to sample from a distribution with mass proportional to $p_0(x_0) p(ζ|x_0)^α$ but fail to achieve this in practice. We introduce a Sequential Monte Carlo algorithm that generates unbiasedly from this target distribution, utilising the learnt unconditional and guided process. We validate our approach on low-dimensional distributions, controlled images and text generations. For text generation, our method provides strong control while maintaining low perplexity compared to guidance-based approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。