arXiv:2601.10403cs.LG2026-01被引 9

让离散扩散模型在生成时灵活控制分布,无需重新训练。

Discrete Feynman-Kac Correctors

  • 基于SMC算法,在推理时动态调节样本分布温度。
  • 可实现退火采样、多过程联合采样及奖励导向生成。
  • 适用于代码生成、蛋白质设计等需精准控制的场景。

离散扩散模型作为生成离散序列的新兴方法,通过渐进去噪或去掩码过程捕捉数据中的层次化非顺序依赖关系。然而,这些定制过程难以灵活控制生成分布。本文提出离散费曼-卡茨校正器(Discrete Feynman-Kac Correctors),在推理阶段对已训练的离散掩码扩散模型进行分布调控。我们推导出序列蒙特卡洛(SMC)算法,可实现:给定训练好的模型,调节采样分布温度(即执行退火)、从多个不同条件扩散过程的边际乘积中采样、或结合外部奖励函数采样,从而生成既符合目标分布又具有高奖励的样本。重要的是,该框架无需额外训练或微调原模型。我们在多个任务中验证其有效性:高效采样伊辛模型的退火玻尔兹曼分布,提升代码生成与变分推断中语言模型性能,以及奖励引导的蛋白质序列生成。

原文摘要 · Abstract (English)

Discrete diffusion models have recently emerged as a promising alternative to the autoregressive approach for generating discrete sequences. Sample generation via gradual denoising or demasking processes allows them to capture hierarchical non-sequential interdependencies in the data. These custom processes, however, do not assume a flexible control over the distribution of generated samples. We propose Discrete Feynman-Kac Correctors, a framework that allows for controlling the generated distribution of discrete masked diffusion models at inference time. We derive Sequential Monte Carlo (SMC) algorithms that, given a trained discrete diffusion model, control the temperature of the sampled distribution (i.e. perform annealing), sample from the product of marginals of several diffusion processes (e.g. differently conditioned processes), and sample from the product of the marginal with an external reward function, producing likely samples from the target distribution that also have high reward. Notably, our framework does not require any training of additional models or fine-tuning of the original model. We illustrate the utility of our framework in several applications including: efficient sampling from the annealed Boltzmann distribution of the Ising model, improving the performance of language models for code generation and amortized learning, as well as reward-tilted protein sequence generation.

扩散模型序列生成奖励驱动

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。