arXiv:2501.00740cs.CV2025-01CVPR被引 24

通过人机协作构建高质量数据集,提升图像去物的准确率与清晰度。

RORem: Training a Robust Object Remover with Human-in-the-Loop

  • 引入人机协同筛选机制,生成超20万对高质量去物训练数据
  • 在200K数据上微调后,去物成功率提升超18%
  • 适合需要高精度图像编辑的视觉任务研究者使用

现有去物方法普遍存在去除不彻底、内容合成错误和生成区域模糊等问题,主要源于缺乏高质量成对训练数据,以及自监督训练导致的修复歧义。为此,本文提出一种人机协同的半监督学习策略:先从开源数据集收集6万对初始训练数据训练基础模型,生成去物样本;再通过人工反馈筛选高质量样本,训练判别器以自动化后续数据生成。经过多轮迭代,最终获得超过20万对的高质量去物数据集。在此基础上微调预训练的Stable Diffusion模型,得到RORem,其在可靠性与图像质量上均达到当前最优,去物成功率较之前方法提升超过18%。相关数据集、代码与模型已开源。

原文摘要 · Abstract (English)

Despite the significant advancements, existing object removal methods struggle with incomplete removal, incorrect content synthesis and blurry synthesized regions, resulting in low success rates. Such issues are mainly caused by the lack of high-quality paired training data, as well as the self-supervised training paradigm adopted in these methods, which forces the model to in-paint the masked regions, leading to ambiguity between synthesizing the masked objects and restoring the background. To address these issues, we propose a semi-supervised learning strategy with human-in-the-loop to create high-quality paired training data, aiming to train a Robust Object Remover (RORem). We first collect 60K training pairs from open-source datasets to train an initial object removal model for generating removal samples, and then utilize human feedback to select a set of high-quality object removal pairs, with which we train a discriminator to automate the following training data generation process. By iterating this process for several rounds, we finally obtain a substantial object removal dataset with over 200K pairs. Fine-tuning the pre-trained stable diffusion model with this dataset, we obtain our RORem, which demonstrates state-of-the-art object removal performance in terms of both reliability and image quality. Particularly, RORem improves the object removal success rate over previous methods by more than 18\%. The dataset, source code and trained model are available at https://github.com/leeruibin/RORem.

图像修复人机协作扩散模型去物

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。