用扩散模型推理被遮挡物体的完整轮廓，提升视频分割准确率。
Using Diffusion Priors for Video Amodal Segmentation
- 将视频遮挡分割建模为条件生成任务，利用时序信息推断物体边界。
- 在四个数据集上实现最高13%的遮挡区域分割精度提升。
- 适合研究视频理解、生成模型应用与视觉推理的开发者参考。
人类具有物体恒常性的认知能力，能理解物体在完全遮挡下的存在性。当前物体分割方法仅关注可见部分，忽视了世界中的非模态特性。现有少数非模态方法中，单图方法难以处理高遮挡场景，而多帧方法仅针对刚性物体。为此，本文将视频非模态分割视为条件生成任务，利用视频生成模型的基础知识。通过以物体的模态掩码序列和上下文伪深度图为条件，使模型学习被遮挡的边界并推断完整物体范围，随后通过内容补全阶段修复遮挡区域。在四个数据集上的实验表明，该方法在物体遮挡区域的分割性能相比现有最优方法最高提升13%。
原文摘要 · Abstract (English)
Object permanence in humans is a fundamental cue that helps in understanding persistence of objects, even when they are fully occluded in the scene. Present day methods in object segmentation do not account for this amodal nature of the world, and only work for segmentation of visible or modal objects. Few amodal methods exist; single-image segmentation methods cannot handle high-levels of occlusions which are better inferred using temporal information, and multi-frame methods have focused solely on segmenting rigid objects. To this end, we propose to tackle video amodal segmentation by formulating it as a conditional generation task, capitalizing on the foundational knowledge in video generative models. Our method is simple; we repurpose these models to condition on a sequence of modal mask frames of an object along with contextual pseudo-depth maps, to learn which object boundary may be occluded and therefore, extended to hallucinate the complete extent of an object. This is followed by a content completion stage which is able to inpaint the occluded regions of an object. We benchmark our approach alongside a wide array of state-of-the-art methods on four datasets and show a dramatic improvement of upto 13% for amodal segmentation in an object's occluded region.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。