通过分段搜索优化去噪步骤与网络结构,显著加速扩散模型生成。
Flexiffusion: Segment-wise Neural Architecture Search for Flexible Denoising Schedule
- 将生成过程分为全步、部分步和空步,灵活组合以减少计算量。
- 在LDM-4-G上实现2.6倍加速,Stable Diffusion V1.5达5.1倍。
- 无需训练即可搜索,适合追求高效生成的部署场景。
扩散模型虽能生成高质量图像,但因需大量串行去噪步骤而计算成本高。现有神经架构搜索(NAS)方法耗时长,需评估数千个模型。本文提出Flexiffusion,一种无需训练的NAS范式,通过将生成过程划分为等距步段,每段包含全步(计算全部模块)、部分步(计算部分模块)和空步(无计算),自动探索各段的灵活组合,大幅降低搜索成本并提升加速效果。所搜模型在原始LDM-4-G上实现2.6倍加速,在当前最优方法上达1.5倍;对Stable Diffusion V1.5,分别实现5.1倍与2.0倍加速。多数据集验证表明,Flexiffusion可有效消除扩散模型冗余。
原文摘要 · Abstract (English)
Diffusion models are cutting-edge generative models adept at producing diverse, high-quality images. Despite their effectiveness, these models often require significant computational resources owing to their numerous sequential denoising steps and the significant inference cost of each step. Recently, Neural Architecture Search (NAS) techniques have been employed to automatically search for faster generation processes. However, NAS for diffusion is inherently time-consuming as it requires estimating thousands of diffusion models to search for the optimal one. In this paper, we introduce Flexiffusion, a novel training-free NAS paradigm designed to accelerate diffusion models by concurrently optimizing generation steps and network structures. Specifically, we partition the generation process into isometric step segments, each sequentially composed of a full step, multiple partial steps, and several null steps. The full step computes all network blocks, while the partial step involves part of the blocks, and the null step entails no computation. Flexiffusion autonomously explores flexible step combinations for each segment, substantially reducing search costs and enabling greater acceleration compared to the state-of-the-art (SOTA) method for diffusion models. Our searched models reported speedup factors of $2.6\times$ and $1.5\times$ for the original LDM-4-G and the SOTA, respectively. The factors for Stable Diffusion V1.5 and the SOTA are $5.1\times$ and $2.0\times$. We also verified the performance of Flexiffusion on multiple datasets, and positive experiment results indicate that Flexiffusion can effectively reduce redundancy in diffusion models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。