提出分层优化框架,让扩散模型采样更快更稳,5次函数调用就达顶尖质量。
Hierarchical Schedule Optimization for Fast and Robust Diffusion Model Sampling
- 分两层迭代优化:先找全局初始策略,再局部精细调整采样步序。
- 仅需5次函数调用(NFE=5),在LAION-Aesthetics上实现FID 11.94的顶尖效果。
- 无需重训练,单次优化耗时不足8秒,适合实际部署的高效加速方案。
扩散概率模型虽生成质量高,但采样过程缓慢。一种无需训练的加速方法是调度优化,旨在固定少量函数调用次数(NFE)下寻找最优时间步分布以提升样本质量。理想方案需满足有效性、自适应性、实用鲁棒性和计算效率四原则。现有方法难以兼顾。为此,本文提出分层调度优化器(HSO),一种新颖高效的双层优化框架。它将全局最优调度搜索转化为可处理的问题:通过交替进行上层全局初始化策略搜索与下层局部调度优化。核心创新包括:中点误差代理(MEP),一种与求解器无关且数值稳定的局部优化目标;间距惩罚适应度(SPF)函数,通过惩罚过于密集的时间步来保障实际鲁棒性。大量实验表明,HSO在极低NFE条件下达到训练免费采样的新基准。例如,当NFE=5时,于Stable Diffusion v2.1在LAION-Aesthetics数据集上实现FID 11.94。该性能不依赖重训练,单次优化成本低于8秒,展现出极强实用性与效率。
原文摘要 · Abstract (English)
Diffusion probabilistic models have set a new standard for generative fidelity but are hindered by a slow iterative sampling process. A powerful training-free strategy to accelerate this process is Schedule Optimization, which aims to find an optimal distribution of timesteps for a fixed and small Number of Function Evaluations (NFE) to maximize sample quality. To this end, a successful schedule optimization method must adhere to four core principles: effectiveness, adaptivity, practical robustness, and computational efficiency. However, existing paradigms struggle to satisfy these principles simultaneously, motivating the need for a more advanced solution. To overcome these limitations, we propose the Hierarchical-Schedule-Optimizer (HSO), a novel and efficient bi-level optimization framework. HSO reframes the search for a globally optimal schedule into a more tractable problem by iteratively alternating between two synergistic levels: an upper-level global search for an optimal initialization strategy and a lower-level local optimization for schedule refinement. This process is guided by two key innovations: the Midpoint Error Proxy (MEP), a solver-agnostic and numerically stable objective for effective local optimization, and the Spacing-Penalized Fitness (SPF) function, which ensures practical robustness by penalizing pathologically close timesteps. Extensive experiments show that HSO sets a new state-of-the-art for training-free sampling in the extremely low-NFE regime. For instance, with an NFE of just 5, HSO achieves a remarkable FID of 11.94 on LAION-Aesthetics with Stable Diffusion v2.1. Crucially, this level of performance is attained not through costly retraining, but with a one-time optimization cost of less than 8 seconds, presenting a highly practical and efficient paradigm for diffusion model acceleration.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。