arXiv:2506.02488cs.CVcs.AI2025-06被引 3

无需重训练,一键优化扩散模型速度与质量

Flexiffusion: Training-Free Segment-Wise Neural Architecture Search for Efficient Diffusion Models

  • 将生成过程拆分为可动态组合的片段,实现无重训架构搜索
  • 在ImageNet和MS-COCO上实现2倍以上加速,FID下降低于5%
  • 使用轻量级rFID评估,节省90%以上计算时间,适合快速部署

扩散模型(DMs)虽能生成高质量图像,但因多步迭代推理导致计算成本高。现有神经架构搜索(NAS)方法受限于需重训练、搜索复杂度指数级增长以及依赖大量图像生成的慢速评估。为此,我们提出Flexiffusion,一种无需重训练的NAS框架,可在不修改预训练参数的前提下联合优化生成调度与模型架构。核心思想是将生成过程分解为等长灵活片段,每个片段动态组合三种步骤类型:全计算、部分缓存复用计算和跳过计算。该段级搜索空间相比逐步优化大幅缩减候选池,同时保持架构多样性。此外,我们引入相对FID(rFID),通过衡量输出与教师模型的偏差而非真实标签来评估性能,使评估时间减少超90%。实际应用中,Flexiffusion在LDMs、Stable Diffusion和DDPMs上均实现至少2倍加速,且FID下降不超过5%,优于已有NAS与缓存方法。尤其在Stable Diffusion上实现5.1倍加速,CLIP分数几乎不变。本工作开创了高效扩散模型搜索的新范式。

原文摘要 · Abstract (English)

Diffusion models (DMs) are powerful generative models capable of producing high-fidelity images but are constrained by high computational costs due to iterative multi-step inference. While Neural Architecture Search (NAS) can optimize DMs, existing methods are hindered by retraining requirements, exponential search complexity from step-wise optimization, and slow evaluation relying on massive image generation. To address these challenges, we propose Flexiffusion, a training-free NAS framework that jointly optimizes generation schedules and model architectures without modifying pre-trained parameters. Our key insight is to decompose the generation process into flexible segments of equal length, where each segment dynamically combines three step types: full (complete computation), partial (cache-reused computation), and null (skipped computation). This segment-wise search space reduces the candidate pool exponentially compared to step-wise NAS while preserving architectural diversity. Further, we introduce relative FID (rFID), a lightweight evaluation metric for NAS that measures divergence from a teacher model's outputs instead of ground truth, slashing evaluation time by over $90\%$. In practice, Flexiffusion achieves at least $2\times$ acceleration across LDMs, Stable Diffusion, and DDPMs on ImageNet and MS-COCO, with FID degradation under $5\%$, outperforming prior NAS and caching methods. Notably, it attains $5.1\times$ speedup on Stable Diffusion with near-identical CLIP scores. Our work pioneers a resource-efficient paradigm for searching high-speed DMs without sacrificing quality.

扩散模型架构搜索高效生成无重训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。