用扩散模型指导解码器,实现更小的潜在空间与高效图像生成。
DGAE: Diffusion-Guided Autoencoder for Efficient Latent Representation Learning

- 用扩散模型引导解码器恢复丢失信息,提升重建质量。
- 在2倍更小的潜在空间下达到顶尖性能,压缩率更高。
- 适合追求高效生成模型的开发者,尤其适配扩散模型加速训练。
自编码器通过视觉标记化将像素压缩到潜在空间,赋能当前最先进的图像与视频生成模型。尽管近期进展缓解了高压缩比下的性能下降问题,但由生成对抗网络(GAN)引发的训练不稳定性仍是未解难题。本文旨在提升空间压缩效率的同时最小化潜在空间维度,实现更高效紧凑的表示。为此,我们聚焦于增强解码器的表达能力,提出扩散引导自编码器(DGAE),利用扩散模型指导解码器恢复未完全解码的有用信号。该设计有效缓解了高空间压缩率下的性能退化。实验表明,DGAE在潜在空间缩小2倍的情况下仍达到领先性能;当与扩散模型结合时,在ImageNet-1K图像生成任务上表现优异,并显著加快扩散模型收敛速度。
原文摘要 · Abstract (English)
Autoencoders empower state-of-the-art image and video generative models by compressing pixels into a latent space through visual tokenization. Although recent advances have alleviated the performance degradation of autoencoders under high compression ratios, addressing the training instability caused by GAN remains an open challenge. While improving spatial compression, we also aim to minimize the latent space dimensionality, enabling more efficient and compact representations. To tackle these challenges, we focus on improving the decoder's expressiveness. Concretely, we propose DGAE, which employs a diffusion model to guide the decoder in recovering informative signals that are not fully decoded from the latent representation. With this design, DGAE effectively mitigates the performance degradation under high spatial compression rates. At the same time, DGAE achieves state-of-the-art performance with a 2x smaller latent space. When integrated with Diffusion Models, DGAE demonstrates competitive performance on image generation for ImageNet-1K and shows that this compact latent representation facilitates faster convergence of the diffusion model.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。