通过结构化潜在空间加速扩散模型收敛，提升生成质量与速度
DC-AE 1.5: Accelerating Diffusion Model Convergence with Structured Latent Space
- 设计结构化潜在空间，前层捕获物体结构，后层保留细节
- 引入增强扩散训练，在物体通道上添加额外目标，加快收敛速度
- 在ImageNet上比原版快4倍，且生成质量更优
我们提出DC-AE 1.5，一种用于高分辨率扩散模型的深度压缩自编码器新家族。增加自编码器潜在通道数能有效提升重建质量，但会导致扩散模型收敛变慢，反而降低生成质量，限制了潜在扩散模型的质量上限，并阻碍使用更高空间压缩比的自编码器。为此，我们引入两项关键创新：i）结构化潜在空间，一种基于训练的方法，在潜在空间中施加特定的通道级结构，使前序通道捕捉物体结构，后序通道保留图像细节；ii）增强扩散训练，一种在物体潜在通道上加入额外扩散训练目标的策略，以加速收敛。实验表明，DC-AE 1.5在收敛速度和扩散扩展性能上均优于原始版本。在ImageNet 512x512数据集上，DC-AE-1.5-f64c128在生成质量上优于DC-AE-f32c32，同时速度快4倍。代码已开源：https://github.com/dc-ai-projects/DC-Gen。
原文摘要 · Abstract (English)
We present DC-AE 1.5, a new family of deep compression autoencoders for high-resolution diffusion models. Increasing the autoencoder's latent channel number is a highly effective approach for improving its reconstruction quality. However, it results in slow convergence for diffusion models, leading to poorer generation quality despite better reconstruction quality. This issue limits the quality upper bound of latent diffusion models and hinders the employment of autoencoders with higher spatial compression ratios. We introduce two key innovations to address this challenge: i) Structured Latent Space, a training-based approach to impose a desired channel-wise structure on the latent space with front latent channels capturing object structures and latter latent channels capturing image details; ii) Augmented Diffusion Training, an augmented diffusion training strategy with additional diffusion training objectives on object latent channels to accelerate convergence. With these techniques, DC-AE 1.5 delivers faster convergence and better diffusion scaling results than DC-AE. On ImageNet 512x512, DC-AE-1.5-f64c128 delivers better image generation quality than DC-AE-f32c32 while being 4x faster. Code: https://github.com/dc-ai-projects/DC-Gen.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。