提出频谱全维建模框架,实现高保真像素级图像生成
FrequencyBooster: Full-Frequency Modeling for High-Fidelity Pixel Diffusion

- 设计高容量解码器与DiT结合,同步捕捉高频细节与低频语义
- 256×256下FID达1.60(320轮),512×512下FID为1.69,刷新纪录
- 避免局部优化牺牲全局结构,适合追求极致画质的生成任务
为突破基于VAE的潜在扩散模型在保真度与优化对齐上的固有瓶颈,像素空间扩散模型作为端到端范式崭露头角。然而现有方法常难以兼顾计算效率与高频细节保留,多依赖块压缩或受限局部解码,导致‘频谱妥协’,抑制高频与精细像素信息。为此,我们提出全新框架FrequencyBooster,无需高昂开销即实现像素扩散的全频建模能力。其核心为一个专精于提取全频细节与低频语义的高容量解码器,后者源自扩散Transformer(DiT)主干网络。不同于以往以牺牲全局上下文换取局部细化的方法,FrequencyBooster利用高维特征表示,在保持全局结构完整的同时实现更优像素级精度。ImageNet上的大量实验表明,本方法效果显著:在256×256分辨率下,仅用320个训练轮次即达状态最优的FID 1.60;在512×512分辨率下,FID达1.69,显著优于现有像素空间与潜在空间生成模型。
原文摘要 · Abstract (English)
To circumvent the inherent fidelity bottlenecks and optimization misalignment of VAE-based latent diffusion, pixel-space diffusion models have emerged as a compelling end-to-end paradigm. However, existing pixel diffusion models often struggle to balance computational efficiency with the preservation of high-frequency details. They frequently resort to patch-based compression or restricted local decoding, leading to a "spectral compromise" where high-frequency and fine-grained pixel information are suppressed. To address these challenges, we propose \textbf{FrequencyBooster}, a novel framework designed to empower pixel diffusion with full-frequency modeling capabilities without prohibitive overhead. The core of our method is a high-capacity decoder that specializes in extracting exhaustive high-frequency details and low-frequency semantics, the latter of which is derived from a Diffusion Transformer (DiT) backbone. Unlike prior works that sacrifice global context for local refinement, FrequencyBooster leverages high-dimensional feature representations to maintain global structural integrity while achieving superior pixel-level precision. Extensive experiments on ImageNet demonstrate the effectiveness of our approach: our model achieves a state-of-the-art FID of \textbf{1.60} at $256 \times 256$ resolution within only 320 epochs. Furthermore, at $512 \times 512$ resolution, FrequencyBooster attains an FID of \textbf{1.69}, significantly outperforming existing pixel-space and latent-space generative models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。