arXiv:2409.18256cs.CV2024-09被引 9

用扩散模型估计物体形状先验,提升被遮挡目标的分割精度

Amodal Instance Segmentation with Diffusion Shape Prior Estimation

  • 基于扩散模型提取形状先验,融合可见部分与遮挡信息
  • 在多个基准上实现优于现有方法的分割效果
  • 适合需要精确分割遮挡物体的视觉任务研究者

非可见实例分割(Amodal Instance Segmentation, AIS)旨在同时预测图像中物体的可见与被遮挡部分。以往方法依赖训练数据中的形状先验,易过拟合且忽略类别细节。本文提出AISDiff,引入扩散形状先验估计(DiffSP)模块。该方法首先预测可见分割掩码与物体类别,并通过遮挡掩码进行遮挡感知处理;随后将这些信息输入DiffSP模块,利用预训练的条件扩散模型从大规模数据中提取丰富的视觉特征,用于形状先验估计。此外,设计了基于注意力的形状先验分割器,进一步优化最终的非可见分割结果。在多个AIS基准上的实验表明,AISDiff显著提升了分割性能。

原文摘要 · Abstract (English)

Amodal Instance Segmentation (AIS) presents an intriguing challenge, including the segmentation prediction of both visible and occluded parts of objects within images. Previous methods have often relied on shape prior information gleaned from training data to enhance amodal segmentation. However, these approaches are susceptible to overfitting and disregard object category details. Recent advancements highlight the potential of conditioned diffusion models, pretrained on extensive datasets, to generate images from latent space. Drawing inspiration from this, we propose AISDiff with a Diffusion Shape Prior Estimation (DiffSP) module. AISDiff begins with the prediction of the visible segmentation mask and object category, alongside occlusion-aware processing through the prediction of occluding masks. Subsequently, these elements are inputted into our DiffSP module to infer the shape prior of the object. DiffSP utilizes conditioned diffusion models pretrained on extensive datasets to extract rich visual features for shape prior estimation. Additionally, we introduce the Shape Prior Amodal Predictor, which utilizes attention-based feature maps from the shape prior to refine amodal segmentation. Experiments across various AIS benchmarks demonstrate the effectiveness of our AISDiff.

实例分割扩散模型遮挡处理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。