无需训练的扩散模型引导方法，提升生成质量与提示对齐
Token Perturbation Guidance for Diffusion Models
- 通过扰动中间令牌表示提供无训练引导信号
- 无条件生成FID降低近2倍，提示对齐接近有指导模型
- 适用于条件与非条件生成，兼容主流扩散模型
Classifier-free guidance（CFG）已成为现代扩散模型提升生成质量和条件对齐的关键组件，但其需特定训练流程且仅限于条件生成。为此，我们提出无需训练的令牌扰动引导（TPG），直接对扩散网络中的中间令牌表示施加扰动矩阵。TPG采用保范洗牌操作，提供稳定有效的引导信号，不改变模型结构即可提升生成质量。该方法无需训练、不受输入条件限制，可广泛应用于条件与非条件生成任务。我们进一步分析了TPG提供的引导项，发现其采样效果更接近于CFG，优于现有无训练引导方法。在SDXL和Stable Diffusion 2.1上的大量实验表明，TPG在无条件生成中相比SDXL基线实现近2倍的FID改善，同时在提示对齐上接近CFG表现。结果证明TPG是一种通用、条件无关的引导方法，能为更广泛的扩散模型带来类似CFG的收益。
原文摘要 · Abstract (English)
Classifier-free guidance (CFG) has become an essential component of modern diffusion models to enhance both generation quality and alignment with input conditions. However, CFG requires specific training procedures and is limited to conditional generation. To address these limitations, we propose Token Perturbation Guidance (TPG), a novel method that applies perturbation matrices directly to intermediate token representations within the diffusion network. TPG employs a norm-preserving shuffling operation to provide effective and stable guidance signals that improve generation quality without architectural changes. As a result, TPG is training-free and agnostic to input conditions, making it readily applicable to both conditional and unconditional generation. We further analyze the guidance term provided by TPG and show that its effect on sampling more closely resembles CFG compared to existing training-free guidance techniques. Extensive experiments on SDXL and Stable Diffusion 2.1 show that TPG achieves nearly a 2$\times$ improvement in FID for unconditional generation over the SDXL baseline, while closely matching CFG in prompt alignment. These results establish TPG as a general, condition-agnostic guidance method that brings CFG-like benefits to a broader class of diffusion models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。