用中间隐变量替代噪声做零样本图像编辑,提升修改精度与原图保真度。
Exploring Iterative Manifold Constraint for Zero-shot Image Editing
- 选取中间隐变量作为编辑起点,平衡可编辑性与结构保留
- 设计交替去噪与反演的ZigZag流程,逐步注入提示引导
- 通过流形迭代约束减少保真误差,适合零样本图像编辑场景
文本驱动图像编辑需兼顾可编辑性与保真度,即编辑区域应匹配目标提示,其余部分保持不变。现有方法多采用“反演-编辑”流程,将输入图像反演至近高斯噪声 ${z}_T$ 后基于目标提示采样。然而我们发现,以 ${z}_T$ 为起点会引入大量保真误差。初步分析表明,中间反演隐变量 ${z}_p$ 在编辑性与保真性之间表现更优。为此提出新范式 ZZEdit:先定位满足编辑需求且结构保留的中间隐变量 ${z}_p$;再通过交替执行去噪与反演的 ZigZag 过程,逐步注入目标引导并保留 $p$ 步结构信息;最后在目标提示下进行纯采样以达成相同步数。本质上,该方法实现了 $M_p$ 与 $M_{p-1}$ 流形间的迭代约束,显著减少保真误差。大量实验表明,相比传统“反演-编辑”流程,ZZEdit 在多种编辑场景中均具更强效果。
原文摘要 · Abstract (English)
Editability and fidelity are two essential demands for text-driven image editing, which expects that the editing area should align with the target prompt and the rest remain unchanged separately. The current cutting-edge editing methods usually obey an "inversion-then-editing" pipeline, where the input image is inverted to an approximate Gaussian noise ${z}_T$, based on which a sampling process is conducted using the target prompt. Nevertheless, we argue that it is not a good choice to use a near-Gaussian noise as a pivot for further editing since it would bring plentiful fidelity errors. We verify this by a pilot analysis, discovering that intermediate-inverted latents can achieve a better trade-off between editability and fidelity than the fully-inverted ${z}_T$. Based on this, we propose a novel zero-shot editing paradigm dubbed ZZEdit, which first locates a qualified intermediate-inverted latent marked as ${z}_p$ as a better editing pivot, which is sufficient-for-editing while structure-preserving. Then, a ZigZag process is designed to execute denoising and inversion alternately, which progressively inject target guidance to ${z}_p$ while preserving the structure information of $p$ step. Afterwards, to achieve the same step number of inversion and denoising, we execute a pure sampling process under the target prompt. Essentially, our ZZEdit performs iterative manifold constraint between the manifold of $M_{p}$ and $M_{p-1}$, leading to fewer fidelity errors. Extensive experiments highlight the effectiveness of ZZEdit in diverse image editing scenarios compared with the "inversion-then-editing" pipeline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。