仅用3步采样实现高质量图像生成,突破扩散模型推理速度瓶颈。
Three Creates All: You Only Sample 3 Steps
- 通过分层时间嵌入优化,让模型在少步采样中更精准去噪。
- 仅3步采样即达当前最佳性能,与轻量级方法差距显著缩小。
- 无需额外计算开销,适配主流生成框架,易部署实用。
扩散模型虽能生成高保真内容,但因需多次顺序网络评估而推理缓慢。我们发现标准时间步条件是少步采样的关键瓶颈。受分层去噪动态启发,提出多层时间嵌入优化(MTEO):冻结预训练扩散主干,从参考轨迹中蒸馏出少量逐步、分层的时间嵌入。MTEO可无缝接入现有ODE求解器,不增加推理开销,仅训练极少量参数。在多种数据集与主干模型上的实验表明,其在少步采样中达到最先进性能,显著缩小了基于蒸馏与轻量方法之间的差距。代码将公开。
原文摘要 · Abstract (English)
Diffusion models deliver high-fidelity generation but remain slow at inference time due to many sequential network evaluations. We find that standard timestep conditioning becomes a key bottleneck for few-step sampling. Motivated by layer-dependent denoising dynamics, we propose Multi-layer Time Embedding Optimization (MTEO), which freeze the pretrained diffusion backbone and distill a small set of step-wise, layer-wise time embeddings from reference trajectories. MTEO is plug-and-play with existing ODE solvers, adds no inference-time overhead, and trains only a tiny fraction of parameters. Extensive experiments across diverse datasets and backbones show state-of-the-art performance in the few-step sampling and substantially narrow the gap between distillation-based and lightweight methods. Code will be available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。