arXiv:2511.19365cs.CVcs.AI2025-11中稿 · CVPR被引 47

将图像高频细节与低频语义分离生成,提升像素扩散模型效率与质量。

DeCo: Frequency-Decoupled Pixel Diffusion for End-to-End Image Generation

  • 分频解耦设计:用轻量编码器生成高频细节,主模型专注低频语义。
  • 在ImageNet上达FID 1.62(256x256)和2.22(512x512),逼近隐空间扩散方法。
  • 适合追求高效高质图像生成的开发者与研究者,尤其关注端到端像素级建模。

像素扩散直接在像素空间端到端生成图像,避免了两阶段潜在扩散中VAE的限制,具备更高模型容量。现有像素扩散模型训练与推理缓慢,因通常在同一扩散Transformer(DiT)中同时建模高频信号与低频语义。为实现更高效的像素扩散范式,本文提出频率解耦像素扩散框架(DeCo)。基于高频与低频成分应分离生成的直觉,我们使用轻量级像素解码器,在DiT提供的语义引导下生成高频细节,从而让DiT专注于低频语义建模。此外,引入一种频率感知流匹配损失,强调视觉显著频率,抑制不重要频率。大量实验表明,DeCo在像素扩散模型中表现优异,在ImageNet上分别取得FID 1.62(256×256)和2.22(512×512)的成绩,接近潜在扩散方法。此外,其预训练文本到图像模型在系统级评估GenEval中取得0.86的领先综合得分。代码已公开于https://github.com/Zehong-Ma/DeCo。

原文摘要 · Abstract (English)

Pixel diffusion aims to generate images directly in pixel space in an end-to-end fashion. This approach avoids the limitations of VAE in the two-stage latent diffusion, offering higher model capacity. Existing pixel diffusion models suffer from slow training and inference, as they usually model both high-frequency signals and low-frequency semantics within a single diffusion transformer (DiT). To pursue a more efficient pixel diffusion paradigm, we propose the frequency-DeCoupled pixel diffusion framework. With the intuition to decouple the generation of high and low frequency components, we leverage a lightweight pixel decoder to generate high-frequency details conditioned on semantic guidance from the DiT. This thus frees the DiT to specialize in modeling low-frequency semantics. In addition, we introduce a frequency-aware flow-matching loss that emphasizes visually salient frequencies while suppressing insignificant ones. Extensive experiments show that DeCo achieves superior performance among pixel diffusion models, attaining FID of 1.62 (256x256) and 2.22 (512x512) on ImageNet, closing the gap with latent diffusion methods. Furthermore, our pretrained text-to-image model achieves a leading overall score of 0.86 on GenEval in system-level comparison. Codes are publicly available at https://github.com/Zehong-Ma/DeCo.

图像生成扩散模型像素扩散频率解耦

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。