用新损失函数提升扩散模型图像编辑的边缘保真度。
Edge-Aware Image Manipulation via Diffusion Models with a Novel Structure-Preservation Loss
- 引入结构保真损失，通过局部线性模型量化编辑前后结构差异。
- 无需训练即可保持边缘细节，在多个任务中达当前最优效果。
- 适合需要高保真图像编辑的研究者与开发者使用。
近期图像编辑研究利用潜在扩散模型（LDMs）实现多种任务下的文本驱动编辑。然而，保持像素级边缘结构——对照片级风格迁移或图像色调调整等任务至关重要——仍是基于潜在扩散模型编辑的挑战。为此，我们提出一种新型结构保真损失（SPL），利用局部线性模型量化输入与编辑后图像之间的结构差异。该无训练方法将SPL直接集成到扩散模型的生成过程中，以确保结构保真。核心机制辅以去噪后处理步骤以减轻解码失真、掩码策略实现精确编辑定位，以及颜色保真损失以保留未编辑区域的色调。实验表明，SPL显著提升了结构保真度，在基于潜在扩散模型的图像编辑中达到当前最优性能。代码将公开于 https://github.com/gongms00/SPL。
原文摘要 · Abstract (English)
Recent advances in image editing leverage latent diffusion models (LDMs) for versatile, text-prompt-driven edits across diverse tasks. Yet, maintaining pixel-level edge structures-crucial for tasks such as photorealistic style transfer or image tone adjustment-remains as a challenge for latent-diffusion-based editing. To overcome this limitation, we propose a novel Structure Preservation Loss (SPL) that leverages local linear models to quantify structural differences between input and edited images. Our training-free approach integrates SPL directly into the diffusion model's generative process to ensure structural fidelity. This core mechanism is complemented by a post-processing step to mitigate LDM decoding distortions, a masking strategy for precise edit localization, and a color preservation loss to preserve hues in unedited areas. Experiments confirm SPL enhances structural fidelity, delivering state-of-the-art performance in latent-diffusion-based image editing. Our code will be publicly released at https://github.com/gongms00/SPL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。