让扩散模型支持多次编辑而不失真，实现多种操作连续叠加。
REED-VAE: RE-Encode Decode Training for Iterative Image Editing with Diffusion Models
- 通过重编码-解码训练，减少每次编辑的噪声累积。
- 在多轮文本/掩码编辑中保持图像质量，成功率显著提升。
- 适合需要连续修改图像的设计师和研究人员使用。
尽管潜在扩散模型在图像编辑上表现优异，但其在对同一图像进行连续编辑时受限严重。现有方法在重复进行像素与潜在空间转换时会积累伪影和噪声。部分方法虽尝试在潜在空间内完成整个编辑链，但仅支持预设的有限编辑操作，牺牲了灵活性。本文提出一种针对变分自编码器（VAE）的重编码-解码（REED）训练方案，有效提升多次迭代后的图像保真度。该方法支持多方法连续编辑：用户可将基于扩散的操作与传统编辑技术结合，逐轮构建输出。我们在文本驱动和掩码驱动等多种编辑框架中验证了REED-VAE的优势，证明其提升了图像的整体可编辑性，显著提高了成功且精确编辑的概率。本工作旨在为多方法图像编辑这一新任务提供基准。代码与模型将公开于https://github.com/galmog/REED-VAE。
原文摘要 · Abstract (English)
While latent diffusion models achieve impressive image editing results, their application to iterative editing of the same image is severely restricted. When trying to apply consecutive edit operations using current models, they accumulate artifacts and noise due to repeated transitions between pixel and latent spaces. Some methods have attempted to address this limitation by performing the entire edit chain within the latent space, sacrificing flexibility by supporting only a limited, predetermined set of diffusion editing operations. We present a RE-encode decode (REED) training scheme for variational autoencoders (VAEs), which promotes image quality preservation even after many iterations. Our work enables multi-method iterative image editing: users can perform a variety of iterative edit operations, with each operation building on the output of the previous one using both diffusion-based operations and conventional editing techniques. We demonstrate the advantage of REED-VAE across a range of image editing scenarios, including text-based and mask-based editing frameworks. In addition, we show how REED-VAE enhances the overall editability of images, increasing the likelihood of successful and precise edit operations. We hope that this work will serve as a benchmark for the newly introduced task of multi-method image editing. Our code and models will be available at https://github.com/galmog/REED-VAE
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。