让AI精准编辑图像,保持人脸身份不变且只改指定区域。
S$^2$Edit: Text-Guided Image Editing with Precise Semantic and Spatial Control
- 用可学习的文本标记嵌入身份信息,通过正交约束分离身份与属性。
- 引入物体掩码控制注意力分布,确保修改仅作用于目标区域。
- 适合需要精细控制的人脸化妆、换装等编辑任务,效果更自然。
扩散模型虽已实现高质量文本引导的图像生成与操控,但直接用于需细粒度控制的编辑任务(如人脸编辑)时,常导致身份信息和高频细节丢失,或因概念纠缠而误改无关区域。本文提出S²Edit,基于预训练文本到图像扩散模型,实现个性化编辑并具备精确语义与空间控制。首先微调模型,将身份信息嵌入可学习文本标记;训练中通过在文本特征空间施加正交性约束,使身份标记与待编辑属性解耦。为确保身份标记仅影响目标区域,采用物体掩码引导交叉注意力图。推理时,方法能局部化编辑,同时忠实保留原始身份,实现语义解耦且空间聚焦的身份标记。大量实验表明,S²Edit在定量与定性指标上均优于现有最优方法。此外,本文还展示了多种组合式图像编辑应用,如妆容迁移。
原文摘要 · Abstract (English)
Recent advances in diffusion models have enabled high-quality generation and manipulation of images guided by texts, as well as concept learning from images. However, naive applications of existing methods to editing tasks that require fine-grained control, e.g., face editing, often lead to suboptimal solutions with identity information and high-frequency details lost during the editing process, or irrelevant image regions altered due to entangled concepts. In this work, we propose S$^2$Edit, a novel method based on a pre-trained text-to-image diffusion model that enables personalized editing with precise semantic and spatial control. We first fine-tune our model to embed the identity information into a learnable text token. During fine-tuning, we disentangle the learned identity token from attributes to be edited by enforcing an orthogonality constraint in the textual feature space. To ensure that the identity token only affects regions of interest, we apply object masks to guide the cross-attention maps. At inference time, our method performs localized editing while faithfully preserving the original identity with semantically disentangled and spatially focused identity token learned. Extensive experiments demonstrate the superiority of S$^2$Edit over state-of-the-art methods both quantitatively and qualitatively. Additionally, we showcase several compositional image editing applications of S$^2$Edit such as makeup transfer.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。