无需训练，通过噪声修正实现高效图像编辑。
Diffusion-Based Image-to-Image Translation by Noise Correction via Prompt Interpolation
- 用提示词插值生成噪声修正项，改进扩散模型预测
- 结合原始去噪与修正项，精准保留不变区域并修改目标区域
- 可无缝集成现有方法，性能提升且延迟极低
我们提出一种简单而有效的免训练方法，用于基于扩散模型的图像到图像转换。该方法通过引入噪声修正项，对预训练扩散模型的原始噪声预测网络进行修正。噪声修正项定义为两个噪声预测的差值：一个来自源与目标提示词嵌入逐步插值后的去噪网络，另一个则使用源提示词嵌入得到。最终的噪声预测由标准去噪项与噪声修正项的线性组合构成，前者用于重建必须保留的区域，后者旨在有效编辑与目标提示相关的目标区域。该方法可轻松融入现有基于扩散模型的图像到图像转换框架中。大量实验表明，所提方法在低延迟下实现卓越性能，并在与现有框架结合时持续提升效果。
原文摘要 · Abstract (English)
We propose a simple but effective training-free approach tailored to diffusion-based image-to-image translation. Our approach revises the original noise prediction network of a pretrained diffusion model by introducing a noise correction term. We formulate the noise correction term as the difference between two noise predictions; one is computed from the denoising network with a progressive interpolation of the source and target prompt embeddings, while the other is the noise prediction with the source prompt embedding. The final noise prediction network is given by a linear combination of the standard denoising term and the noise correction term, where the former is designed to reconstruct must-be-preserved regions while the latter aims to effectively edit regions of interest relevant to the target prompt. Our approach can be easily incorporated into existing image-to-image translation methods based on diffusion models. Extensive experiments verify that the proposed technique achieves outstanding performance with low latency and consistently improves existing frameworks when combined with them.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。