通过直接对齐噪声域,实现更精准的文本引导图像编辑。
DNAEdit: Direct Noise Alignment for Text-Guided Rectified Flow Editing
- 直接在噪声空间修正高斯噪声,减少误差累积。
- 在长提示下编辑效果优于当前最优方法,背景保留更佳。
- 适合需要精确控制图像生成与编辑的开发者和研究者。
利用大规模预训练文本到图像模型的强大生成能力,无需训练的方法已展现出优异的图像编辑效果。传统基于扩散的方法以及近期基于修正流(RF)的方法通常通过逐步向干净图像添加噪声来反向合成轨迹,过程中当前时间步的噪声潜在表示用于近似下一时间步,导致误差累积并降低重建精度。考虑到在修正流中,噪声潜在表示是通过在每个时间步直接插值高斯噪声与干净图像得到的,我们提出直接噪声对齐(DNA),直接在噪声域中精炼期望的高斯噪声,显著减少先前方法中的误差累积。具体而言,DNA估计每个时间步插值噪声潜在表示的速度场,并通过计算预测速度场与期望速度场之间的差异来调整高斯噪声。我们验证了DNA的有效性,并揭示其与现有基于修正流的反演方法的关系。此外,我们引入移动速度引导(MVG)以控制目标提示引导的生成过程,在保持图像背景和目标对象可编辑性之间取得平衡。DNA与MVG共同构成所提出的DNAEdit方法。最后,我们引入DNA-Bench,一个长提示基准测试,用于评估先进图像编辑模型的性能。实验结果表明,我们的DNAEdit在多个指标上均优于当前最先进的文本引导编辑方法。代码与基准测试将发布于 https://xiechenxi99.github.io/DNAEdit/。
原文摘要 · Abstract (English)
Leveraging the powerful generation capability of large-scale pretrained text-to-image models, training-free methods have demonstrated impressive image editing results. Conventional diffusion-based methods, as well as recent rectified flow (RF)-based methods, typically reverse synthesis trajectories by gradually adding noise to clean images, during which the noisy latent at the current timestep is used to approximate that at the next timesteps, introducing accumulated drift and degrading reconstruction accuracy. Considering the fact that in RF the noisy latent is estimated through direct interpolation between Gaussian noises and clean images at each timestep, we propose Direct Noise Alignment (DNA), which directly refines the desired Gaussian noise in the noise domain, significantly reducing the error accumulation in previous methods. Specifically, DNA estimates the velocity field of the interpolated noised latent at each timestep and adjusts the Gaussian noise by computing the difference between the predicted and expected velocity field. We validate the effectiveness of DNA and reveal its relationship with existing RF-based inversion methods. Additionally, we introduce a Mobile Velocity Guidance (MVG) to control the target prompt-guided generation process, balancing image background preservation and target object editability. DNA and MVG collectively constitute our proposed method, namely DNAEdit. Finally, we introduce DNA-Bench, a long-prompt benchmark, to evaluate the performance of advanced image editing models. Experimental results demonstrate that our DNAEdit achieves superior performance to state-of-the-art text-guided editing methods. Codes and benchmark will be available at \href{ https://xiechenxi99.github.io/DNAEdit/}{https://xiechenxi99.github.io/DNAEdit/}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。