让扩散模型直接从噪声生成数据,提升生成质量和效率。
E2ED^2:Direct Mapping from Noise to Data for Enhanced Diffusion Models
- 直接从噪声到数据端到端训练,消除中间步骤偏差。
- 仅需少于4步采样,FID和CLIP分数显著提升。
- 可融合对抗与感知损失,适合追求高效高质生成的场景。
扩散模型已成为视觉生成领域的主流范式,在图像合成到时序视频生成等众多应用中表现卓越。然而仍存在三大根本问题:训练与推理过程不一致、噪声污染过程中的信息逐级泄露、难以有效融入感知或对抗性损失等现代优化目标。为此,本文提出一种全新的端到端学习框架E2ED^2,实现从初始噪声到目标数据分布的直接优化。该方法通过将训练视为从各向同性高斯噪声到数据分布的直接映射,消除了训练-采样不匹配与中间信息泄露。同时,该框架可无缝集成对抗损失与感知损失至核心优化目标。在COCO30K与HW30K等标准基准上的全面评估表明,本方法在仅使用少于4次采样步数的情况下,即显著提升弗雷谢特起始距离(FID)与CLIP分数,验证了端到端机制在结合扩散模型稳定性与GAN式判别优化方面的潜力。
原文摘要 · Abstract (English)
Diffusion models have established themselves as the de facto primary paradigm in visual generative modeling, revolutionizing the field through remarkable success across various diverse applications ranging from high-quality image synthesis to temporal aware video generation. Despite these advancements, three fundamental limitations persist, including 1) discrepancy between training and inference processes, 2) progressive information leakage throughout the noise corruption procedures, and 3) inherent constraints preventing effective integration of modern optimization criteria like perceptual and adversarial loss. To mitigate these critical challenges, we in this paper present a novel end-to-end learning paradigm that establishes direct optimization from the final generated samples to initial noises. Our proposed End-to-End Differentiable Diffusion, dubbed E2ED^2, introduces several key improvements: it eliminates the sequential training-sampling mismatch and intermediate information leakage via conceptualizing training as a direct transformation from isotropic Gaussian noise to the target data distribution. Additionally, such training framework enables seamless incorporation of adversarial and perceptual losses into the core optimization objective. Comprehensive evaluation across standard benchmarks including COCO30K and HW30K reveals that our method achieves substantial performance gains in terms of Fréchet Inception Distance (FID) and CLIP score, even with fewer sampling steps (less than 4). Our findings highlight that the end-to-end mechanism might pave the way for more robust and efficient solutions, \emph{i.e.,} combining diffusion stability with GAN-like discriminative optimization in an end-to-end manner.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。