MagicEraser用扩散模型实现精准物体擦除，生成自然背景。
MagicEraser: Erasing Any Objects via Semantics-Aware Control
- 分两阶段：先初始化内容，再通过提示调优和语义注意力重聚焦控制生成。
- 在COCO、Cityscapes数据集上实现高保真背景重建，避免生成错位纹理。
- 适合需要精细图像编辑的设计师与内容创作者使用。
传统图像修复任务旨在通过参考周围背景和前景来恢复受损区域。然而，日益增长的物体擦除任务目标是移除物体并生成协调一致的背景。基于GAN的修复方法在复杂纹理生成方面表现不佳；新兴的基于扩散模型的方法（如Stable Diffusion Inpainting）虽能生成新内容，但常在被擦除位置产生不一致结果，且依赖高质量文本提示。为此，我们提出MagicEraser——一种专为物体擦除任务设计的扩散模型框架。其包含两个阶段：内容初始化与可控生成。在第二阶段，我们引入两个即插即用模块：提示调优（prompt tuning）与语义感知注意力重聚焦（semantics-aware attention refocus）。此外，我们设计了一种数据构建策略，生成特别适用于该任务的训练数据。实验表明，MagicEraser在内容生成上实现了精细有效的控制，显著减少了不良伪影，在物体擦除任务中取得实质性进展。
原文摘要 · Abstract (English)
The traditional image inpainting task aims to restore corrupted regions by referencing surrounding background and foreground. However, the object erasure task, which is in increasing demand, aims to erase objects and generate harmonious background. Previous GAN-based inpainting methods struggle with intricate texture generation. Emerging diffusion model-based algorithms, such as Stable Diffusion Inpainting, exhibit the capability to generate novel content, but they often produce incongruent results at the locations of the erased objects and require high-quality text prompt inputs. To address these challenges, we introduce MagicEraser, a diffusion model-based framework tailored for the object erasure task. It consists of two phases: content initialization and controllable generation. In the latter phase, we develop two plug-and-play modules called prompt tuning and semantics-aware attention refocus. Additionally, we propose a data construction strategy that generates training data specially suitable for this task. MagicEraser achieves fine and effective control of content generation while mitigating undesired artifacts. Experimental results highlight a valuable advancement of our approach in the object erasure task.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。