统一处理物体删除与插入，实现精准无痕图像编辑。
OmniPaint: Mastering Object-Oriented Editing via Disentangled Insertion-Removal Inpainting
- 将删物与插物视为互相关联任务，利用扩散先验建模
- 在未配对数据上通过CycleFlow实现大规模精调，提升效果
- 提出CFD指标评估一致性，适合高保真图像编辑场景
基于扩散的生成模型已革新物体导向图像编辑，但真实场景下的物体删除与插入仍受物理效应复杂性和缺乏成对训练数据的制约。本文提出OmniPaint，一种统一框架，将物体删除与插入重新构想为相互依赖的过程而非独立任务。该框架结合预训练扩散先验与渐进式训练流程：先通过初始成对样本优化，再利用CycleFlow在大规模未配对数据上进行精细化微调，实现精确前景消除与无缝物体插入，同时忠实保留场景几何结构与固有属性。此外，我们提出新型参考无关评估指标CFD，用于衡量上下文一致性和物体幻觉程度，建立高保真图像编辑新基准。
原文摘要 · Abstract (English)
Diffusion-based generative models have revolutionized object-oriented image editing, yet their deployment in realistic object removal and insertion remains hampered by challenges such as the intricate interplay of physical effects and insufficient paired training data. In this work, we introduce OmniPaint, a unified framework that re-conceptualizes object removal and insertion as interdependent processes rather than isolated tasks. Leveraging a pre-trained diffusion prior along with a progressive training pipeline comprising initial paired sample optimization and subsequent large-scale unpaired refinement via CycleFlow, OmniPaint achieves precise foreground elimination and seamless object insertion while faithfully preserving scene geometry and intrinsic properties. Furthermore, our novel CFD metric offers a robust, reference-free evaluation of context consistency and object hallucination, establishing a new benchmark for high-fidelity image editing. Project page: https://yeates.github.io/OmniPaint-Page/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。