arXiv:2510.12586cs.CV2025-10被引 21

无需变分自编码器,直接在像素空间训练出顶级生成模型。

There is No VAE: End-to-End Pixel-Space Generative Modeling via Self-Supervised Pre-training

  • 先预训练编码器捕捉语义并对齐采样轨迹,再端到端微调完整模型。
  • 像素空间扩散模型在ImageNet-256上达FID 1.58,仅需75次函数评估。
  • 首次实现无VAE的高分辨率一致性模型,性能超越潜在空间方法。

像素空间生成模型通常更难训练且性能较差,与潜在空间模型存在显著性能与效率差距。本文提出一种两阶段训练框架,成功缩小这一差距,适用于像素空间扩散与一致性模型。第一阶段预训练编码器,从干净图像中捕捉有意义语义,并对齐沿同一确定性采样轨迹的点,该轨迹从先验分布演化至数据分布。第二阶段将预训练编码器与随机初始化解码器结合,端到端微调完整模型。该框架在ImageNet上达到当前最优性能:扩散模型在ImageNet-256上取得FID 1.58,ImageNet-512上为2.35,仅需75次函数评估(NFE),显著优于以往像素空间方法及基于VAE的方法,在生成质量与训练效率上均大幅领先。直接对比显示,该模型性能远超DiT,仅使用其约30%的训练算力。此外,一致性模型在ImageNet-256上实现FID 8.82,显著超越其潜在空间对手。这是首个不依赖预训练VAE或扩散模型、直接在高分辨率图像上训练的一致性模型。代码已开源。

原文摘要 · Abstract (English)

Pixel-space generative models are often more difficult to train and generally underperform compared to their latent-space counterparts, leaving a persistent performance and efficiency gap. In this paper, we introduce a novel two-stage training framework that closes this gap for pixel-space diffusion and consistency models. In the first stage, we pre-train encoders to capture meaningful semantics from clean images while aligning them with points along the same deterministic sampling trajectory, which evolves points from the prior to the data distribution. In the second stage, we integrate the encoder with a randomly initialized decoder and fine-tune the complete model end-to-end for both diffusion and consistency models. Our framework achieves state-of-the-art (SOTA) performance on ImageNet. Specifically, our diffusion model reaches an FID of 1.58 on ImageNet-256 and 2.35 on ImageNet-512 with 75 number of function evaluations (NFE) surpassing prior pixel-space methods and VAE-based counterparts by a large margin in both generation quality and training efficiency. In a direct comparison, our model significantly outperforms DiT while using only around 30\% of its training compute. Furthermore, our consistency model achieves an impressive FID of 8.82 on ImageNet-256, significantly outperforming its latent-space counterparts. This marks the first successful training of a consistency model directly on high-resolution images without relying on pre-trained VAEs or diffusion models. Our codes are available at: \href{https://github.com/AMAP-ML/EPG}{https://github.com/AMAP-ML/EPG}

生成模型像素空间扩散模型一致性模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。