用改进的ConvNeXt设计高效扩散模型，训练快7倍且节省算力。
Reviving ConvNeXt for Efficient Convolutional Diffusion Models
- 基于ConvNeXt结构设计全卷积扩散模型，保留局部性与参数效率
- 仅需DiT-XL/2一半算力，256×256下训练步数减少7倍
- 4卡即可训练，适合资源受限的高效生成任务
近期扩散模型多采用Transformer骨干网络，因其全注意力架构具有出色可扩展性。然而，卷积网络固有的局部性偏好、参数效率及硬件友好性在现代生成建模中未被充分探索。本文提出全卷积扩散模型（FCDM），其骨干结构类似ConvNeXt，专为条件扩散建模设计。结果显示，仅使用DiT-XL/2 50%的浮点运算量（FLOPs），FCDM-XL在256×256和512×512分辨率下分别实现7倍和7.5倍的训练步数减少。尤为关键的是，FCDM-XL可在4张GPU上完成训练，凸显其卓越的训练效率。结果表明，现代卷积设计为扩散模型提供了极具竞争力且高效的替代方案，使ConvNeXt重新成为高效生成建模的简洁而强大的基础模块。
原文摘要 · Abstract (English)
Recent diffusion models increasingly favor Transformer backbones, motivated by the remarkable scalability of fully attentional architectures. Yet the locality bias, parameter efficiency, and hardware friendliness--the attributes that established ConvNets as the efficient vision backbone--have seen limited exploration in modern generative modeling. Here we introduce the fully convolutional diffusion model (FCDM), a model having a backbone similar to ConvNeXt, but designed for conditional diffusion modeling. We find that using only 50% of the FLOPs of DiT-XL/2, FCDM-XL achieves competitive performance with 7$\times$ and 7.5$\times$ fewer training steps at 256$\times$256 and 512$\times$512 resolutions, respectively. Remarkably, FCDM-XL can be trained on a 4-GPU system, highlighting the exceptional training efficiency of our architecture. Our results demonstrate that modern convolutional designs provide a competitive and highly efficient alternative for scaling diffusion models, reviving ConvNeXt as a simple yet powerful building block for efficient generative modeling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。