用二值化生成模型实现高精度图像编辑,避免传统方法的反演误差。
EditInfinity: Image Editing with Binary-Quantized Generative Models
- 利用二值化生成模型的精确中间表示,实现更精准的图像反演。
- 在PIE-Bench上三项编辑任务均优于现有扩散模型基线。
- 适合需要高保真与语义精准的图像编辑场景。
将预训练的基于扩散的生成模型用于文本驱动的图像编辑,仅需极少调优即可展现巨大潜力。传统方法通过图像反演推导生成轨迹,再依据目标文本提示沿轨迹编辑。然而,图像反演过程中的近似误差严重限制了编辑效果,源于扩散模型中间步骤缺乏精确监督。为解决此问题,我们研究了二值化生成模型的参数高效适配,利用其可获取源图像精确中间量化表示的特性,实现更有效的监督以实现精准图像反演。具体提出EditInfinity,适配二值化生成模型Infinity。设计了一种高效且有效的图像反演机制,结合文本提示修正与图像风格保持,实现精确反演。进一步提出整体平滑策略,使EditInfinity能以高保真度保留源图细节并精确对齐文本语义。在PIE-Bench基准上针对'add'、'change'、'delete'三类编辑操作的大量实验表明,本模型性能显著优于现有扩散模型基线。代码已开源:https://github.com/yx-chen-ust/EditInfinity。
原文摘要 · Abstract (English)
Adapting pretrained diffusion-based generative models for text-driven image editing with negligible tuning overhead has demonstrated remarkable potential. A classical adaptation paradigm, as followed by these methods, first infers the generative trajectory inversely for a given source image by image inversion, then performs image editing along the inferred trajectory guided by the target text prompts. However, the performance of image editing is heavily limited by the approximation errors introduced during image inversion by diffusion models, which arise from the absence of exact supervision in the intermediate generative steps. To circumvent this issue, we investigate the parameter-efficient adaptation of binary-quantized generative models for image editing, and leverage their inherent characteristic that the exact intermediate quantized representations of a source image are attainable, enabling more effective supervision for precise image inversion. Specifically, we propose EditInfinity, which adapts \emph{Infinity}, a binary-quantized generative model, for image editing. We propose an efficient yet effective image inversion mechanism that integrates text prompting rectification and image style preservation, enabling precise image inversion. Furthermore, we devise a holistic smoothing strategy which allows our EditInfinity to perform image editing with high fidelity to source images and precise semantic alignment to the text prompts. Extensive experiments on the PIE-Bench benchmark across `add', `change', and `delete' editing operations, demonstrate the superior performance of our model compared to state-of-the-art diffusion-based baselines. Code available at: https://github.com/yx-chen-ust/EditInfinity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。