用数学方法实现无需训练的精准图像编辑。
h-Edit: Effective and Flexible Diffusion-Based Editing via Doob's h-Transform
- 基于扩散模型反向过程构建编辑路径，分解更新为重建与编辑两部分。
- 支持文本和奖励模型双重引导，可同时处理复杂编辑任务。
- 无需重新训练，适合快速、灵活的图像修改需求。
我们提出一个扩散图像编辑的理论框架，将编辑问题建模为反向时间桥接过程。该方法通过修改预训练扩散模型的反向过程，构建一条在时间0收敛到目标编辑分布的桥接路径。基于此框架，我们提出h-Edit方法，利用Doob's h-transform与Langevin Monte Carlo，将中间样本的更新分解为“重建”项与“编辑”项。这一分解赋予灵活性：重建项可由现有反演技术计算，多个编辑项可组合以应对复杂编辑任务。据我们所知，h-Edit是首个无需训练即可实现文本引导与奖励模型联合编辑的方法。大量定量与定性实验表明，h-Edit在编辑效果与忠实度上均优于当前最优基线。代码已开源：https://github.com/nktoan/h-edit。
原文摘要 · Abstract (English)
We introduce a theoretical framework for diffusion-based image editing by formulating it as a reverse-time bridge modeling problem. This approach modifies the backward process of a pretrained diffusion model to construct a bridge that converges to an implicit distribution associated with the editing target at time 0. Building on this framework, we propose h-Edit, a novel editing method that utilizes Doob's h-transform and Langevin Monte Carlo to decompose the update of an intermediate edited sample into two components: a "reconstruction" term and an "editing" term. This decomposition provides flexibility, allowing the reconstruction term to be computed via existing inversion techniques and enabling the combination of multiple editing terms to handle complex editing tasks. To our knowledge, h-Edit is the first training-free method capable of performing simultaneous text-guided and reward-model-based editing. Extensive experiments, both quantitative and qualitative, show that h-Edit outperforms state-of-the-art baselines in terms of editing effectiveness and faithfulness. Our source code is available at https://github.com/nktoan/h-edit.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。