提出扩散模型蒸馏的定量近似框架,解析低噪声下得分逼近与动态不稳定的权衡。
A Quantitative Approximation Framework for Flow Distillation in Diffusion Models
- 将少步采样视为学习流映射的组合近似,构建定量分析框架。
- 在高斯混合模型中证明了ReLU/ReQU网络在时间均匀的L^p空间内可实现得分近似,复杂度为多项式对数级。
- 通过非均匀分段降低误差51.9%,适合追求高效低延迟扩散模型部署的研究者。
我们通过将少步采样视为学习流映射的组合近似,构建了扩散模型蒸馏的定量分析框架。针对概率流ODE的轨迹蒸馏,发现低噪声多模态情形下得分逼近能力与动力学稳定性分离:尽管得分仍可高效逼近,但小的局部误差可能被刚性流动力学显著放大。在高斯混合Ornstein–Uhlenbeck模型中,我们证明了使用ReLU和ReQU网络可在时间均匀的L^p(p_t)空间中实现得分近似,且复杂度为显式的多项式对数级,并推导出流速的可计算Lipschitz界L(t)。稳定因子𝑧𝑚𝑠𝑟𝑤𝑞𝑥(∫∫s^t L(u)𝑝 u)随噪声降低和混合成分分离而指数增长。将此证书与单步学生模型的局部Lipschitz预算对比,识别出直接蒸馏困难的区域,但不意味着逼近下限。我们还证明深层残差结构通过传播局部误差控制全局传输误差,且均衡累积稳定性可实现最优非均匀分割。采用八段网格时,最终均相对均方误差最多降低51.9%相较于均匀网格。
原文摘要 · Abstract (English)
We develop a quantitative framework for diffusion distillation by viewing few step sampling as approximation through compositions of learned flow maps. For trajectory distillation of the probability flow ODE, we show that low noise multimodal regimes separate score approximability from dynamical stability: the score remains efficiently approximable, while small local errors may be strongly amplified by stiff flow dynamics. In a Gaussian mixture Ornstein--Uhlenbeck model, we prove time uniform \(L^p(p_t)\) score approximation by ReLU and ReQU networks with explicit polylogarithmic complexity, and derive a computable Lipschitz bound \(L(t)\) for the flow velocity. The stability factor \(\exp\bigl(\int_s^t L(u)\mathrm du\bigr)\) can grow exponentially as noise decreases and mixture separation increases. Comparing this certificate with a certified local Lipschitz budget for one step students identifies regimes of direct distillation difficulty, without implying an approximation lower bound. We also show that deep residual compositions control global transport error through propagated local errors, and that equalizing cumulative stability yields an optimal nonuniform segmentation. With eight segments, this grid reduces final mean relative MSE by up to \(51.9\%\) versus uniform grids.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。