改进扩散模型少步采样质量，通过建模逆过程协方差提升清晰度。
Covariance-aware sampling for Diffusion Models
- 基于逆过程协方差设计采样器，不再仅依赖均值预测
- 在相同函数评估次数下，优于现有二阶采样器和aDDIM
- 仅需每步多一次JVP计算，实现低开销高效优化
我们提出一种协方差感知采样器，显著提升像素空间扩散模型在少步采样场景下的生成质量。我们假设少步采样失败源于仅依赖逆过程的均值预测，而我们的方法显式建模逆过程协方差。通过结合Tweedie公式估计协方差，并采用高效的结构化傅里叶空间分解，该方法可作为DDIM的扩展实现。每步仅需额外一次雅可比-向量积（JVP），计算开销极小。实验表明，在相同函数评估次数（NFE）下，该方法对像素级扩散模型生成的样本质量持续优于当前最优的二阶采样器（Heun、DPM-Solver++）以及近期的aDDIM采样器。
原文摘要 · Abstract (English)
We present a covariance-aware sampler that improves the quality of pixel-space Diffusion Model (DM) sampling in the few-step regime. We hypothesize that in the few-step regime samplers fail because they rely solely on the predicted mean of the reverse distribution, while our solution explicitly models the reverse-process covariance. Our method combines Tweedie's formula to estimate the covariance with an efficient, structured Fourier-space decomposition of the covariance matrix. Implemented as an extension of DDIM, our method requires only a minimal overhead: one extra Jacobian-Vector Product (JVP) per step. We demonstrate that for pixel-based DMs, our method consistently produces superior samples compared to state-of-the-art second order samplers (Heun, DPM-Solver++) and the recent aDDIM sampler, at an identical number of function evaluations (NFE).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。