将修复与删除任务分离,提升特定类别物体移除效果。
Task-Decoupled Image Inpainting Framework for Class-specific Object Remover
- 拆分修复与删除任务,分别训练专用模型。
- 在多个数据集上显著优于通用图像修复网络。
- 适合需要精准移除特定类物体的场景。
物体移除旨在消除图像中的指定对象并保持整体外观。现有方法使用图像修复网络进行移除,但结果常不理想。本文发现,当前训练方式让单一模型同时处理修复与移除任务,是导致效果不佳的原因。为此,提出一种任务解耦的图像修复框架,生成两个独立模型:用于修复的物体修复器和用于移除的物体移除器。通过部分覆盖目标区域的掩码训练修复器,再利用其生成的输出作为指导,训练移除器。该框架构建了聚焦特定类别的物体移除器,可更有效地移除目标类别物体。同时提出数据整理方法,涵盖图像筛选与掩码生成策略,模拟具备真实移除标注的数据训练场景。实验表明,在多个数据集上,所提类别特定移除器在移除目标类别物体方面优于基于图像修复网络的通用移除器。
原文摘要 · Abstract (English)
Object removal refers to the process of erasing designated objects from an image while preserving the overall appearance. Existing works on object removal erase removal targets using image inpainting networks. However, image inpainting networks often generate unsatisfactory removal results. In this work, we find that the current training approach which encourages a single image inpainting model to handle both object removal and restoration tasks is one of the reasons behind such unsatisfactory result. Based on this finding, we propose a task-decoupled image inpainting framework which generates two separate inpainting models: an object restorer for object restoration tasks and an object remover for object removal tasks. We train the object restorer with the masks that partially cover the removal targets. Then, the proposed framework makes an object restorer to generate a guidance for training the object remover. Using the proposed framework, we obtain a class-specific object remover which focuses on removing objects of a target class, aiming to better erase target class objects than general object removers. We also introduce a data curation method that encompasses the image selection and mask generation approaches used to produce training data for the proposed class-specific object remover. Using the proposed curation method, we can simulate the scenarios where an object remover is trained on the data with object removal ground truth images. Experiments on multiple datasets show that the proposed class-specific object remover can better remove target class objects than object removers based on image inpainting networks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。