用对比损失实现正负概念引导，提升扩散模型采样质量
ContrastiveCFG: Guiding Diffusion Sampling by Contrasting Positive and Negative Concepts
- 通过对比损失控制去噪方向，正向对齐条件，反向排斥无关特征
- 在复杂文本提示下仍能有效注入或移除特定概念，保持生成质量
- 适合需要精准控制生成内容的场景，如艺术创作与图像编辑
Classifier-Free Guidance（CFG）在条件扩散模型采样中已证明能有效提升条件对齐效果。许多应用使用负向引导（NP）来过滤样本中的不想要特征，但简单地反转CFG会生成倒置的概率分布，常导致样本偏离边缘分布。受条件扩散模型用于逆问题的最新进展启发，本文提出一种新方法：利用对比损失实现对给定条件的引导。具体而言，其引导项通过对比损失使去噪方向根据条件对齐或排斥，对正向条件达到与传统CFG相似的引导效果，同时克服现有负向引导方法的局限性。实验表明，该方法在从简单类别条件到复杂重叠文本提示等多种场景中，均能有效注入或移除指定概念，且保持样本质量。
原文摘要 · Abstract (English)
As Classifier-Free Guidance (CFG) has proven effective in conditional diffusion model sampling for improved condition alignment, many applications use a negated CFG term as a Negative Prompting (NP) to filter out unwanted features from samples. However, simply negating CFG guidance creates an inverted probability distribution, often distorting samples away from the marginal distribution. Inspired by recent advances in conditional diffusion models for inverse problems, here we present a novel method to achieve guidance toward the given condition using contrastive loss. Specifically, our guidance term aligns or repels the denoising direction based on the given condition through contrastive loss, achieving a similar guiding effect to traditional CFG for positive conditions while overcoming the limitations of existing negative guidance methods. Experimental results demonstrate that our approach effectively injects or removes the given concepts while maintaining sample quality across diverse scenarios, from simple class conditions to complex and overlapping text prompts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。