无需额外计算，利用训练数据免费获取协方差信息提升扩散模型生成效果。
Free Hunch: Denoiser Covariance Estimation for Diffusion Models Without Extra Costs
- 利用训练数据和生成轨迹曲率，免费获取协方差估计。
- 在低步数条件下优于现有基线方法，尤其在逆问题上表现突出。
- 适合需要快速生成且不改架构的场景，如实时图像修复。
在无需训练的扩散模型引导生成中，给定噪声观测时干净数据的协方差是一个关键量。现有方法需大量推理时间计算、修改标准训练流程或改变去噪器结构，或做严重近似。本文提出新框架，通过利用训练数据中天然存在的协方差信息及生成轨迹曲率（与协方差通过二阶Tweedie公式关联），避免上述问题。我们结合两种信息源：(i) 一种新颖的跨噪声水平协方差传递方法；(ii) 在固定噪声水平下的低秩更新机制。在线性逆问题上验证，该方法在较少扩散步数下显著优于近期基线。
原文摘要 · Abstract (English)
The covariance for clean data given a noisy observation is an important quantity in many training-free guided generation methods for diffusion models. Current methods require heavy test-time computation, altering the standard diffusion training process or denoiser architecture, or making heavy approximations. We propose a new framework that sidesteps these issues by using covariance information that is available for free from training data and the curvature of the generative trajectory, which is linked to the covariance through the second-order Tweedie's formula. We integrate these sources of information using (i) a novel method to transfer covariance estimates across noise levels and (ii) low-rank updates in a given noise level. We validate the method on linear inverse problems, where it outperforms recent baselines, especially with fewer diffusion steps.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。