让图像生成的潜在空间更适应旋转缩放，提升生成速度与质量
EQ-VAE: Equivariance Regularized Latent Space for Improved Generative Image Modeling
- 通过正则化强制潜在空间对语义不变变换保持等变性
- 在仅5轮微调下使DiT-XL/2生成速度提升7倍，重建质量不降
- 适配连续与离散编码器，通用性强，适合各类生成模型优化
潜变量生成模型已成为高质量图像合成的主流方法。这类模型依赖自编码器将图像压缩至潜在空间，再由生成模型学习潜在分布。我们发现现有自编码器缺乏对缩放、旋转等语义保持变换的等变性，导致潜在空间复杂，影响生成性能。为此，我们提出EQ-VAE，一种简单正则化方法，在不降低重建质量的前提下，强制潜在空间具备等变性，从而降低其复杂度。通过用EQ-VAE微调预训练自编码器，我们显著提升了多种前沿生成模型（包括DiT、SiT、REPA和MaskGIT）的性能，使DiT-XL/2在仅5个周期的SD-VAE微调后实现7倍加速。EQ-VAE兼容连续与离散自编码器，为广泛潜变量生成模型提供通用增强方案。
原文摘要 · Abstract (English)
Latent generative models have emerged as a leading approach for high-quality image synthesis. These models rely on an autoencoder to compress images into a latent space, followed by a generative model to learn the latent distribution. We identify that existing autoencoders lack equivariance to semantic-preserving transformations like scaling and rotation, resulting in complex latent spaces that hinder generative performance. To address this, we propose EQ-VAE, a simple regularization approach that enforces equivariance in the latent space, reducing its complexity without degrading reconstruction quality. By finetuning pre-trained autoencoders with EQ-VAE, we enhance the performance of several state-of-the-art generative models, including DiT, SiT, REPA and MaskGIT, achieving a 7 speedup on DiT-XL/2 with only five epochs of SD-VAE fine-tuning. EQ-VAE is compatible with both continuous and discrete autoencoders, thus offering a versatile enhancement for a wide range of latent generative models. Project page and code: https://eq-vae.github.io/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。