用对抗监督让扩散模型生成更准,训练推理对齐更稳。
ADT: Tuning Diffusion Models with Adversarial Supervision
- 引入对抗判别器,让生成过程在训练中模拟真实推理路径。
- 在多个Stable Diffusion版本上显著提升图像质量和分布对齐度。
- 轻量判别器+反向梯度约束,避免内存爆炸且防止模型欺骗判别器。
扩散模型通过逆向加噪过程逼近真实数据分布,在训练中单次前向预测加噪样本的扩散得分,而推理需从白噪声迭代去噪。这种训练与推理的差异导致推理分布与训练分布不一致,源于预测偏差和误差累积。为此,我们提出一种直观有效的微调框架——对抗扩散调优(ADT),通过在优化过程中激励推理过程,并利用对抗监督使最终输出对齐训练数据分布。具体地,ADT采用双塔结构判别器,固定预训练主干网络并仅训练少量参数;引入图像到图像采样策略以平滑判别难度;保留原始扩散损失以防判别器被模型“攻破”。此外,我们谨慎约束反向传播路径,沿推理路径回传梯度而不引发内存溢出或梯度爆炸。大量实验表明,ADT在Stable Diffusion v1.5、XL和v3上均显著提升分布对齐性与图像质量。
原文摘要 · Abstract (English)
Diffusion models have achieved outstanding image generation by reversing a forward noising process to approximate true data distributions. During training, these models predict diffusion scores from noised versions of true samples in a single forward pass, while inference requires iterative denoising starting from white noise. This training-inference divergences hinder the alignment between inference and training data distributions, due to potential prediction biases and cumulative error accumulation. To address this problem, we propose an intuitive but effective fine-tuning framework, called Adversarial Diffusion Tuning (ADT), by stimulating the inference process during optimization and aligning the final outputs with training data by adversarial supervision. Specifically, to achieve robust adversarial training, ADT features a siamese-network discriminator with a fixed pre-trained backbone and lightweight trainable parameters, incorporates an image-to-image sampling strategy to smooth discriminative difficulties, and preserves the original diffusion loss to prevent discriminator hacking. In addition, we carefully constrain the backward-flowing path for back-propagating gradients along the inference path without incurring memory overload or gradient explosion. Finally, extensive experiments on Stable Diffusion models (v1.5, XL, and v3), demonstrate that ADT significantly improves both distribution alignment and image quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。