让图像编辑更精准:通过可调控噪声图实现高保真修改
Editable Noise Map Inversion: Encoding Target-image into Noise For High-Fidelity Image Manipulation
- 设计可编辑的噪声图反演方法,提升文本引导编辑灵活性
- 在多种编辑任务中同时实现高内容保真度与提示匹配度
- 适用于图像和视频编辑,保持帧间一致性
文本到图像扩散模型在生成高质量、多样化图像方面取得了显著进展。在此基础上,扩散模型在文本引导的图像编辑任务中也表现出色。有效编辑的关键策略是将源图像反演为与目标图像相关的可编辑噪声图。然而,现有反演方法难以严格遵循目标文本提示,原因在于反演得到的噪声图虽能忠实重建源图像,却限制了所需编辑的灵活性。为此,我们提出可编辑噪声图反演(ENM Inversion),一种新型反演技术,旨在搜索最优噪声图以兼顾内容保留与编辑能力。通过对噪声图可编辑性特性的分析,我们的方法引入可编辑噪声优化,通过最小化重构噪声图与编辑目标噪声图之间的差异,实现与期望编辑对齐。大量实验表明,ENM Inversion 在广泛图像编辑任务中均优于现有方法,在内容保真度与提示匹配度方面表现更优。该方法还可轻松扩展至视频编辑,实现帧间一致性与跨帧内容操控。
原文摘要 · Abstract (English)
Text-to-image diffusion models have achieved remarkable success in generating high-quality and diverse images. Building on these advancements, diffusion models have also demonstrated exceptional performance in text-guided image editing. A key strategy for effective image editing involves inverting the source image into editable noise maps associated with the target image. However, previous inversion methods face challenges in adhering closely to the target text prompt. The limitation arises because inverted noise maps, while enabling faithful reconstruction of the source image, restrict the flexibility needed for desired edits. To overcome this issue, we propose Editable Noise Map Inversion (ENM Inversion), a novel inversion technique that searches for optimal noise maps to ensure both content preservation and editability. We analyze the properties of noise maps for enhanced editability. Based on this analysis, our method introduces an editable noise refinement that aligns with the desired edits by minimizing the difference between the reconstructed and edited noise maps. Extensive experiments demonstrate that ENM Inversion outperforms existing approaches across a wide range of image editing tasks in both preservation and edit fidelity with target prompts. Our approach can also be easily applied to video editing, enabling temporal consistency and content manipulation across frames.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。