改进扩散模型图文编辑，提升提示词准确性和背景保留效果。
LUSD: Localized Update Score Distillation for Text-Guided Image Editing
- 引入注意力空间正则与梯度滤波归一化，减少梯度波动。
- 在提示词匹配度上超越现有方法，成功编辑率更高。
- 适合需要精准控制图像编辑的AI艺术创作与设计场景。
尽管扩散模型在给定目标提示下实现图像编辑展现出良好效果，但同时保证提示词准确性和背景完整性仍具挑战。现有基于梯度蒸馏的方法利用文本到图像扩散模型丰富的生成先验，在无需额外微调的情况下解决此问题，但在对象插入等任务中表现不佳。我们分析发现，梯度幅值和空间分布存在显著差异，导致超参数调整高度依赖输入或失效。为此，我们提出两项简单而有效的改进：基于注意力的空间正则化与梯度滤波-归一化，旨在降低梯度更新过程中的变异。实验表明，本方法在提示词保真度上优于当前最优的梯度蒸馏技术，提升了成功编辑率并有效保留背景。用户评估显示，本方法在三项指标上均优于现有技术，整体偏好度提升58%-64%。
原文摘要 · Abstract (English)
While diffusion models show promising results in image editing given a target prompt, achieving both prompt fidelity and background preservation remains difficult. Recent works have introduced score distillation techniques that leverage the rich generative prior of text-to-image diffusion models to solve this task without additional fine-tuning. However, these methods often struggle with tasks such as object insertion. Our investigation of these failures reveals significant variations in gradient magnitude and spatial distribution, making hyperparameter tuning highly input-specific or unsuccessful. To address this, we propose two simple yet effective modifications: attention-based spatial regularization and gradient filtering-normalization, both aimed at reducing these variations during gradient updates. Experimental results show our method outperforms state-of-the-art score distillation techniques in prompt fidelity, improving successful edits while preserving the background. Users also preferred our method over state-of-the-art techniques across three metrics, and by 58-64% overall.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。