arXiv:2410.07273cs.CVcs.LG2024-10NeurIPS被引 34

提出新型精确反演采样器,实现扩散模型高质量逆向生成。

BELM: Bidirectional Explicit Linear Multi-step Sampler for Exact Inversion in Diffusion Models

  • 基于双向显式线性多步框架,统一现有启发式方法。
  • 最小化局部截断误差,提升反演精度与采样质量。
  • 适用于图像编辑与插值,理论稳定且收敛性佳。

扩散模型的反演问题旨在寻找样本对应的初始噪声,对多种任务至关重要。尽管已有若干无训练的启发式精确反演采样器被提出,但其理论性质不明,且采样质量普遍不佳。本文提出一种通用框架——双向显式线性多步(BELM)采样器,包含所有已有启发式方法作为特例。该框架源于变步长、变公式线性多步法,并引入双向显式约束,证明此约束是数学上精确反演的关键。我们系统分析了贝尔姆框架内的局部截断误差(LTE),发现现有设计导致次优的LTE。因此,通过最小化LTE提出最优贝尔姆(O-BELM)采样器。进一步分析证实其理论稳定性与全局收敛性。大量实验表明,O-BELM在保持精确反演的同时,实现了高质量采样;在图像编辑与插值任务中的附加实验也展示了其广泛应用潜力。

原文摘要 · Abstract (English)

The inversion of diffusion model sampling, which aims to find the corresponding initial noise of a sample, plays a critical role in various tasks. Recently, several heuristic exact inversion samplers have been proposed to address the inexact inversion issue in a training-free manner. However, the theoretical properties of these heuristic samplers remain unknown and they often exhibit mediocre sampling quality. In this paper, we introduce a generic formulation, \emph{Bidirectional Explicit Linear Multi-step} (BELM) samplers, of the exact inversion samplers, which includes all previously proposed heuristic exact inversion samplers as special cases. The BELM formulation is derived from the variable-stepsize-variable-formula linear multi-step method via integrating a bidirectional explicit constraint. We highlight this bidirectional explicit constraint is the key of mathematically exact inversion. We systematically investigate the Local Truncation Error (LTE) within the BELM framework and show that the existing heuristic designs of exact inversion samplers yield sub-optimal LTE. Consequently, we propose the Optimal BELM (O-BELM) sampler through the LTE minimization approach. We conduct additional analysis to substantiate the theoretical stability and global convergence property of the proposed optimal sampler. Comprehensive experiments demonstrate our O-BELM sampler establishes the exact inversion property while achieving high-quality sampling. Additional experiments in image editing and image interpolation highlight the extensive potential of applying O-BELM in varying applications.

扩散模型反演采样图像生成优化算法

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。