提出DiP框架，在像素空间实现高效高质图像生成。
DiP: Taming Diffusion Models in Pixel Space
- 分阶段生成：用大块结构建模+轻量细节修复
- 推理速度提升10倍，参数仅增0.3%
- 在ImageNet上达1.79的FID，适合高分辨率生成任务
扩散模型在生成质量与计算效率间存在根本权衡。潜在扩散模型（LDM）虽高效，但存在信息损失和非端到端训练问题；现有像素空间模型虽避免了变分自编码器（VAE），但高分辨率生成计算开销巨大。为此，我们提出DiP，一种高效的像素空间扩散框架。DiP将生成过程解耦为全局与局部两个阶段：基于扩散变换器（DiT）的大块处理用于高效构建整体结构，同时协同训练的轻量级补丁细节头利用上下文特征恢复精细局部细节。该协同设计在无需VAE的前提下，达到与LDM相当的计算效率。DiP实现较此前方法最高10倍的推理加速，总参数仅增加0.3%，并在ImageNet 256×256上取得1.79的FID分数。
原文摘要 · Abstract (English)
Diffusion models face a fundamental trade-off between generation quality and computational efficiency. Latent Diffusion Models (LDMs) offer an efficient solution but suffer from potential information loss and non-end-to-end training. In contrast, existing pixel space models bypass VAEs but are computationally prohibitive for high-resolution synthesis. To resolve this dilemma, we propose DiP, an efficient pixel space diffusion framework. DiP decouples generation into a global and a local stage: a Diffusion Transformer (DiT) backbone operates on large patches for efficient global structure construction, while a co-trained lightweight Patch Detailer Head leverages contextual features to restore fine-grained local details. This synergistic design achieves computational efficiency comparable to LDMs without relying on a VAE. DiP is accomplished with up to 10$\times$ faster inference speeds than previous method while increasing the total number of parameters by only 0.3%, and achieves an 1.79 FID score on ImageNet 256$\times$256.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。