通过重要性加权与最优提议设计，实现离散扩散模型推理时的可控生成。
Inference-Time Scaling of Discrete Diffusion Models via Importance Weighting and Optimal Proposal Design
- 基于序列蒙特卡洛框架，设计可扩展的推理控制机制。
- 在文本到图像、生物序列等任务中提升生成质量和可控性。
- 适合需要高精度生成控制的研究者与工程师使用。
离散扩散模型已在多个领域表现出色。然而，实际应用常需生成过程满足特定约束。为此，我们提出一种序列蒙特卡洛（SMC）框架，通过有原则的重要性加权与最优提议构造，实现离散扩散模型的可扩展推理时控制。具体而言，该方法推导出一系列中间目标的可计算重要性权重，并刻画了最优提议；我们进一步提出了两种实用近似：一阶梯度逼近和通过最小化重要性权重对数方差训练的摊销提议。在合成任务、语言建模、生物序列设计及文本到图像生成中的实证结果表明，该框架显著提升了生成可控性和样本质量，凸显了SMC作为离散扩散模型推理时可扩展性的通用方案的有效性。
原文摘要 · Abstract (English)
Discrete diffusion models have become highly effective across various domains. However, real-world applications often require the generative process to adhere to certain constraints. To this end, we propose a Sequential Monte Carlo (SMC) framework that enables scalable inference-time control of discrete diffusion models through principled importance weighting and optimal proposal construction. Specifically, our approach derives tractable importance weights for a range of intermediate targets and characterises the optimal proposal, for which we develop two practical approximations: a first-order gradient-based approximation and an amortised proposal trained to minimise the log-variance of the importance weights. Empirical results across synthetic tasks, language modelling, biology design, and text-to-image generation demonstrate that our framework enhances controllability and sample quality, highlighting the effectiveness of SMC as a versatile recipe for scaling discrete diffusion models at inference time.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。