提出新VAE架构,让图像生成更精细可控。
Boosting Latent Diffusion Models via Disentangled Representation Alignment
- 设计非线性映射结构,使潜在空间具备解耦语义能力。
- 在ImageNet上生成质量达FID 1.21,优于现有方法。
- 适合追求细节保留与属性控制的图像生成研究者。
潜在扩散模型(LDMs)依赖变分自编码器(VAEs)提供的压缩潜在空间实现高质量图像生成。近期研究尝试通过直接借鉴LDM训练中的对齐策略,利用视觉基础模型(VFMs)作为表征对齐目标来获取生成友好型VAE。然而,此类对齐范式忽视了LDMs与VAEs在表征需求上的根本差异:从局部块到高维语义的简单映射会导致语义坍缩,造成细粒度属性丢失。本文揭示关键洞见:相较于依赖高层全局语义的LDMs,生成友好型VAE需具备强语义解耦能力,以结构化方式保留属性级信息。为此,提出语义解耦VAE(Send-VAE)。不同于以往浅层对齐方法,Send-VAE引入非线性映射架构,有效连接VAEs的局部结构与VFMs的密集语义,从而在无显式正则化下激发潜在空间的解耦特性。大量实验建立基于低级属性可分离性的VAE潜在空间评估新范式,并证明Send-VAE在ImageNet 256x256上达到当前最优生成质量(FID 1.21)。
原文摘要 · Abstract (English)
Latent Diffusion Models (LDMs) rely heavily on the compressed latent space provided by Variational Autoencoders (VAEs) for high-quality image generation. Recent studies have attempted to obtain generation-friendly VAEs by directly adopting alignment strategies from LDM training, leveraging Vision Foundation Models (VFMs) as representation alignment targets. However, such alignment paradigms overlook the fundamental differences in representational requirements between LDMs and VAEs. Simple feature mapping from local patches to high-dimensional semantics can induce semantic collapse, leading to the loss of fine-grained attributes. In this paper, we reveal a key insight: unlike LDMs that benefit from high-level global semantics, a generation-friendly VAE must possess strong semantic disentanglement capabilities to preserve fine-grained, attribute-level information in a structured manner. To address this discrepancy, we propose the Semantic-Disentangled VAE (Send-VAE). Deviating from previous shallow alignment approaches, Send-VAE introduces a non-linear mapping architecture to effectively bridge the local structures of VAEs and the dense semantics of VFMs, thereby encouraging emergent disentangled properties in the latent space without explicit regularization. Extensive experiments establish a new paradigm for evaluating VAE latent spaces via low-level attribute separability and demonstrate that Send-VAE achieves state-of-the-art generation quality (FID of 1.21) on ImageNet 256x256.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。