通过优化漂移项的Lipschitz常数,设计更高效的生成模型插值路径。
Lipschitz-Guided Design of Interpolation Schedules in Generative Models
- 以平均平方Lipschitz度为准则,优化生成模型的插值路径设计。
- 对高斯与高斯混合分布目标,实现比线性路径更优的收敛性能。
- 可直接迁移至不同训练路径,无需重新训练,适合高效采样场景。
本文从统计与数值角度研究流模型与扩散模型中的插值路径设计。在随机插值框架下,我们证明经后验优化扩散系数后,标量插值路径在路径空间的KL散度下统计等价。这促使我们关注漂移场的数值性质而非仅统计标准。提出以最小化漂移场的平均平方Lipschitz常数作为路径设计的合理准则,区别于最优传输中的动能最小化。给出一个简洁的转移公式,使所设计路径可在推理时用于原为其他(如线性)路径训练的模型,无需重训练。针对高斯与高斯混合目标,解析求得最优路径:对高斯分布,获得指数级降低的Lipschitz常数;对高斯混合,缓解少步采样中的模式坍缩。在随机Allen-Cahn与Navier-Stokes方程的高维不变测度上验证,所提路径在固定积分器预算下显著提升细尺度统计精度。
原文摘要 · Abstract (English)
We study the design of interpolation schedules in flow and diffusion-based generative models from both statistical and numerical perspectives. Within the stochastic interpolants framework, we first show that scalar interpolation schedules are statistically equivalent under the Kullback--Leibler divergence in path space, after optimal a posteriori tuning of the diffusion coefficient. This equivalence motivates focusing on numerical properties of the drift field rather than purely statistical criteria. We propose minimizing the averaged squared Lipschitzness of the drift as a principled criterion for schedule design, in contrast with kinetic-energy minimization in optimal transport. A simple transfer formula expresses the drift of one schedule in terms of the drift of another, allowing the designed schedule to be used at inference time with a model trained under a different (e.g., linear) schedule, without retraining. We work out the optimal schedules analytically for Gaussian and Gaussian-mixture targets: for Gaussians, we obtain exponential improvements in the Lipschitz constant over linear schedules; for Gaussian mixtures, we obtain schedules that mitigate mode collapse in few-step sampling. We then validate the approach on high-dimensional invariant measures of stochastic Allen--Cahn and Navier--Stokes equations, where the designed schedule yields markedly more accurate fine-scale statistics at fixed integrator budget.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。