用掩码自编码器改进图像生成的潜在扩散模型
Latent Diffusion Models with Masked AutoEncoders
- 提出VMAE，利用掩码自编码器保持层级特征
- 新模型在潜在平滑性、感知压缩和重建质量上全面优于旧方法
- 适合研究图像生成与潜在表示的学者参考
尽管潜在扩散模型（LDMs）在图像生成方面展现出巨大潜力，但其自编码器的设计特性与最优结构仍缺乏深入探索。本文分析了自编码器在LDM中的作用，识别出三个关键属性：潜在平滑性、感知压缩质量和重建质量。我们发现现有自编码器无法同时满足这三项要求，并提出变分掩码自编码器（VMAE），利用掩码自编码器保持的层级特征。将VMAE集成到LDM框架中，构建了基于掩码自编码器的潜在扩散模型（LDMAE）。代码已开源。
原文摘要 · Abstract (English)
In spite of the remarkable potential of Latent Diffusion Models (LDMs) in image generation, the desired properties and optimal design of the autoencoders have been underexplored. In this work, we analyze the role of autoencoders in LDMs and identify three key properties: latent smoothness, perceptual compression quality, and reconstruction quality. We demonstrate that existing autoencoders fail to simultaneously satisfy all three properties, and propose Variational Masked AutoEncoders (VMAEs), taking advantage of the hierarchical features maintained by Masked AutoEncoders. We integrate VMAEs into the LDM framework, introducing Latent Diffusion Models with Masked AutoEncoders (LDMAEs). Our code is available at https://github.com/isno0907/ldmae.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。