用遮挡关系推断实例的3D前后顺序,提升分割精度。
Occlusion-Ordered Semantic Instance Segmentation
- 从遮挡边界同时提取实例与前后顺序
- 在KINS和COCOA数据集上优于强基线
- 适合需要3D结构理解的视觉任务
标准语义实例分割仅提供单张图像的二维信息。为支持三维分析,通常需结合绝对单目深度估计与实例分割,但单目深度估计难度大。本文改用更简单的基于遮挡的相对深度排序,该方法虽粗略但更可靠。提出联合求解相对深度排序与实例分割的任务,称为遮挡有序语义实例分割(OOSIS)。通过定向遮挡边界与语义分割,同时提取实例及其遮挡顺序。不同于主流检测-分割框架,本方法将OOSIS建模为标签问题,实现简洁统一。开发了新型定向遮挡边界方法,显著优于先前工作。还设计了融合实例掩码准确率与遮挡顺序正确性的新评估指标。在KINS与COCOA数据集上表现优于多个强基线。
原文摘要 · Abstract (English)
Standard semantic instance segmentation provides useful, but inherently 2D information from a single image. To enable 3D analysis, one usually integrates absolute monocular depth estimation with instance segmentation. However, monocular depth is a difficult task. Instead, we leverage a simpler single-image task, occlusion-based relative depth ordering, providing coarser but useful 3D information. We show that relative depth ordering works more reliably from occlusions than from absolute depth. We propose to solve the joint task of relative depth ordering and segmentation of instances based on occlusions. We call this task Occlusion-Ordered Semantic Instance Segmentation (OOSIS). We develop an approach to OOSIS that extracts instances and their occlusion order simultaneously from oriented occlusion boundaries and semantic segmentation. Unlike popular detect-and-segment framework for instance segmentation, combining occlusion ordering with instance segmentation allows a simple and clean formulation of OOSIS as a labeling problem. As a part of our solution for OOSIS, we develop a novel oriented occlusion boundaries approach that significantly outperforms prior work. We also develop a new joint OOSIS metric based both on instance mask accuracy and correctness of their occlusion order. We achieve better performance than strong baselines on KINS and COCOA datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。