像素空间扩散模型达1.5 FID，性能超越主流隐空间模型。
Simpler Diffusion (SiD2): 1.5 FID on ImageNet512 with pixel-space diffusion
- 采用简化架构与信噪比加权损失，直接训练像素空间扩散模型。
- 在ImageNet512上实现1.5 FID，刷新图像生成质量纪录。
- 适合追求高效、简洁且高质图像生成的开发者与研究者。
隐空间扩散模型已成为高分辨率图像合成中扩展扩散模型的主流选择。相比端到端训练的像素空间模型，隐空间模型通常被认为更高效且生成质量更高。本文挑战这一观点，证明像素空间模型在质量和效率上同样具备竞争力，能在ImageNet512上达到1.5 FID，并在ImageNet128、ImageNet256和Kinetics600上取得新最优结果。我们提出一种简单可扩展的像素空间扩散模型训练方案：1）使用预设超参数的sigmoid损失加权（Kingma & Gao, 2023）；2）采用简化且内存高效的架构，减少跳接连接；3）模型规模向高分辨率处理倾斜，而非在低分辨率上堆参数。结合指导间隔策略，构建出名为Simpler Diffusion (SiD2) 的像素空间扩散模型家族。
原文摘要 · Abstract (English)
Latent diffusion models have become the popular choice for scaling up diffusion models for high resolution image synthesis. Compared to pixel-space models that are trained end-to-end, latent models are perceived to be more efficient and to produce higher image quality at high resolution. Here we challenge these notions, and show that pixel-space models can be very competitive to latent models both in quality and efficiency, achieving 1.5 FID on ImageNet512 and new SOTA results on ImageNet128, ImageNet256 and Kinetics600. We present a simple recipe for scaling end-to-end pixel-space diffusion models to high resolutions. 1: Use the sigmoid loss-weighting (Kingma & Gao, 2023) with our prescribed hyper-parameters. 2: Use our simplified memory-efficient architecture with fewer skip-connections. 3: Scale the model to favor processing the image at a high resolution with fewer parameters, rather than using more parameters at a lower resolution. Combining these with guidance intervals, we obtain a family of pixel-space diffusion models we call Simpler Diffusion (SiD2).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。