让扩散模型直接生成像素图，效率媲美潜空间模型。
Latent Forcing: Reordering the Diffusion Trajectory for Pixel-Space Image Generation
- 用双路径噪声调度联合处理潜变量与像素，重排去噪顺序。
- 在ImageNet上以同等算力达到扩散变换器像素生成新纪录。
- 适合关注高保真图像生成与模型效率的科研人员。
潜空间扩散模型虽能生成高质量图像，但牺牲了端到端建模优势：图像编码过程丢弃信息，需独立训练解码器，并建模辅助分布而非原始数据。本文提出潜变量强制（Latent Forcing），对现有架构进行简单修改，在保持潜空间模型效率的同时直接操作原始自然图像。该方法通过分别调整噪声调度，联合处理潜变量与像素，有序重排去噪轨迹，使潜变量作为中间计算的临时存储区，待高频像素特征生成前完成推理。我们发现条件信号顺序至关重要，并据此解释了REPA分词器与扩散模型间的蒸馏差异、有条件与无条件生成的区别，以及分词器重建质量与可扩散性之间的关系。应用于ImageNet时，潜变量强制在相同算力下实现了基于扩散变换器的像素生成新最佳表现。
原文摘要 · Abstract (English)
Latent diffusion models excel at generating high-quality images but lose the benefits of end-to-end modeling. They discard information during image encoding, require a separately trained decoder, and model an auxiliary distribution to the raw data. In this paper, we propose Latent Forcing, a simple modification to existing architectures that achieves the efficiency of latent diffusion while operating on raw natural images. Our approach orders the denoising trajectory by jointly processing latents and pixels with separately tuned noise schedules. This allows the latents to act as a scratchpad for intermediate computation before high-frequency pixel features are generated. We find that the order of conditioning signals is critical, and we analyze this to explain differences between REPA distillation in the tokenizer and the diffusion model, conditional versus unconditional generation, and how tokenizer reconstruction quality relates to diffusability. Applied to ImageNet, Latent Forcing achieves a new state-of-the-art for diffusion transformer-based pixel generation at our compute scale.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。