arXiv:2504.19506cs.CV2025-04被引 3

用文本控制实现任意物体的可见区域补全,数据与模型协同生成高质量结果。

SynergyAmodal: Deocclude Anything with Text Control

  • 通过自监督学习将图像修复模型转为部分补全模型,利用真实场景数据提升多样性。
  • 构建人机协同流水线,生成1.6万对高质量配对数据,保证补全结果合理且逼真。
  • 基于合成数据训练可文本控制的完整补全模型,支持零样本泛化和精准编辑。

图像去遮挡(或非可见补全)旨在恢复被遮挡目标的不可见区域(如形状和外观)。尽管近期取得进展,但高质量、兼具多样性、合理性与保真度的数据仍稀缺。为此,我们识别出三个关键要素:利用真实场景图像数据提升多样性,引入人类专家确保合理性,借助生成先验保障保真度。提出SynergyAmodal框架,通过人-数据-模型三者协同,联合生成具有完整形状与外观标注的非可见数据集。首先,设计基于遮挡的自监督学习算法,将图像修复扩散模型微调为部分补全扩散模型;其次,建立迭代式协同合成流程,对部分补全结果进行筛选、优化、选择与标注,在人类专家指导和先验模型约束下保证合理性与保真度;最终生成包含约16,000对样本的高质量配对数据集。最后,在该数据集上训练全补全扩散模型,并以文本提示作为条件信号。大量实验表明,该框架能实现零样本泛化与文本可控性。代码、数据集与模型将公开于https://github.com/imlixinyang/SynergyAmodal。

原文摘要 · Abstract (English)

Image deocclusion (or amodal completion) aims to recover the invisible regions (\ie, shape and appearance) of occluded instances in images. Despite recent advances, the scarcity of high-quality data that balances diversity, plausibility, and fidelity remains a major obstacle. To address this challenge, we identify three critical elements: leveraging in-the-wild image data for diversity, incorporating human expertise for plausibility, and utilizing generative priors for fidelity. We propose SynergyAmodal, a novel framework for co-synthesizing in-the-wild amodal datasets with comprehensive shape and appearance annotations, which integrates these elements through a tripartite data-human-model collaboration. First, we design an occlusion-grounded self-supervised learning algorithm to harness the diversity of in-the-wild image data, fine-tuning an inpainting diffusion model into a partial completion diffusion model. Second, we establish a co-synthesis pipeline to iteratively filter, refine, select, and annotate the initial deocclusion results of the partial completion diffusion model, ensuring plausibility and fidelity through human expert guidance and prior model constraints. This pipeline generates a high-quality paired amodal dataset with extensive category and scale diversity, comprising approximately 16K pairs. Finally, we train a full completion diffusion model on the synthesized dataset, incorporating text prompts as conditioning signals. Extensive experiments demonstrate the effectiveness of our framework in achieving zero-shot generalization and textual controllability. Our code, dataset, and models will be made publicly available at https://github.com/imlixinyang/SynergyAmodal.

图像补全扩散模型文本控制数据合成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。