arXiv:2502.04050cs.CV2025-02International Conf…被引 9

用文本提示精准编辑图像中物体的局部,让扩散模型更懂细节。

PartEdit: Fine-Grained Image Editing using Pre-Trained Diffusion Models

  • 通过优化特殊文本标记,让模型学会定位物体部件
  • 在用户研究中80%以上时间被选为最佳编辑方案
  • 适合需要精细控制图像局部修改的研究者与设计师

我们提出首个基于预训练扩散模型的文本驱动物体局部编辑方法。现有扩散模型对物体局部理解不足,难以实现精细编辑。为此,我们通过高效标记优化过程学习对应不同物体部件的特殊文本标识,使其在推理过程中生成可靠的定位掩码,精准划定编辑区域。结合特征融合与自适应阈值策略,实现无缝编辑。我们建立了一个针对局部编辑的基准数据集与评估协议。实验表明,本方法在所有指标上均优于现有方法,用户研究中66%-90%的场景下更受青睐。

原文摘要 · Abstract (English)

We present the first text-based image editing approach for object parts based on pre-trained diffusion models. Diffusion-based image editing approaches capitalized on the deep understanding of diffusion models of image semantics to perform a variety of edits. However, existing diffusion models lack sufficient understanding of many object parts, hindering fine-grained edits requested by users. To address this, we propose to expand the knowledge of pre-trained diffusion models to allow them to understand various object parts, enabling them to perform fine-grained edits. We achieve this by learning special textual tokens that correspond to different object parts through an efficient token optimization process. These tokens are optimized to produce reliable localization masks at each inference step to localize the editing region. Leveraging these masks, we design feature-blending and adaptive thresholding strategies to execute the edits seamlessly. To evaluate our approach, we establish a benchmark and an evaluation protocol for part editing. Experiments show that our approach outperforms existing editing methods on all metrics and is preferred by users 66-90% of the time in conducted user studies.

图像编辑扩散模型细粒度

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。