用解码器特征设计感知损失，让生成图像更清晰真实
Boosting Latent Diffusion with Perceptual Objectives
- 利用解码器内部特征定义潜空间感知损失
- 在256和512分辨率下FID提升6%至20%
- 适配主流扩散模型，提升细节与真实感
潜空间扩散模型（LDM）是当前高分辨率图像生成的主流方法。它们在自编码器（AE）的潜空间中学习数据分布，并通过AE解码器将生成的潜变量映射到RGB图像空间。然而，这种设计导致扩散模型训练与解码器之间存在脱节，造成生成图像细节丢失。为此，我们提出一种潜空间感知损失（LPL），利用解码器内部特征引导模型生成更清晰、更真实的图像。该损失可无缝集成于常见的自动编码器架构，并适用于不同生成范式，如基于ε预测和速度预测的DDPM，以及流匹配。在三个数据集上以256和512分辨率训练的模型实验表明，使用该感知损失后，定量指标（FID）提升6%至20%，定性效果也显著改善。
原文摘要 · Abstract (English)
Latent diffusion models (LDMs) power state-of-the-art high-resolution generative image models. LDMs learn the data distribution in the latent space of an autoencoder (AE) and produce images by mapping the generated latents into RGB image space using the AE decoder. While this approach allows for efficient model training and sampling, it induces a disconnect between the training of the diffusion model and the decoder, resulting in a loss of detail in the generated images. To remediate this disconnect, we propose to leverage the internal features of the decoder to define a latent perceptual loss (LPL). This loss encourages the models to create sharper and more realistic images. Our loss can be seamlessly integrated with common autoencoders used in latent diffusion models, and can be applied to different generative modeling paradigms such as DDPM with epsilon and velocity prediction, as well as flow matching. Extensive experiments with models trained on three datasets at 256 and 512 resolution show improved quantitative -- with boosts between 6% and 20% in FID -- and qualitative results when using our perceptual loss.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。