通过3D切分语义,实现无需标注的2D实例分割新方法
CutS3D: Cutting Semantics in 3D for 2D Unsupervised Instance Segmentation
- 利用场景点云在3D中切割语义掩码,解决2D重叠实例难分离问题
- 提出空间重要性函数,强化3D边界处的语义清晰度
- 引入三种空间置信度组件,提升无监督检测器学习信号质量
传统2D图像实例分割算法严重依赖大量人工标注数据。近期虽出现无监督方法,但多先生成伪掩码再训练类无关检测器,常因仅考虑语义而无法正确分离2D空间中重叠的实例。为此,本文提出在3D点云表示的场景中切割语义掩码,以获得最终2D实例。同时引入空间重要性函数,沿3D实例边界重新锐化语义。尽管伪掩码仍存在模糊性,我们进一步设计三种空间置信度组件,用于增强类无关检测器的训练,以隔离干净的学习信号。该方法在多个标准无监督实例分割与目标检测基准上优于现有方法。
原文摘要 · Abstract (English)
Traditionally, algorithms that learn to segment object instances in 2D images have heavily relied on large amounts of human-annotated data. Only recently, novel approaches have emerged tackling this problem in an unsupervised fashion. Generally, these approaches first generate pseudo-masks and then train a class-agnostic detector. While such methods deliver the current state of the art, they often fail to correctly separate instances overlapping in 2D image space since only semantics are considered. To tackle this issue, we instead propose to cut the semantic masks in 3D to obtain the final 2D instances by utilizing a point cloud representation of the scene. Furthermore, we derive a Spatial Importance function, which we use to resharpen the semantics along the 3D borders of instances. Nevertheless, these pseudo-masks are still subject to mask ambiguity. To address this issue, we further propose to augment the training of a class-agnostic detector with three Spatial Confidence components aiming to isolate a clean learning signal. With these contributions, our approach outperforms competing methods across multiple standard benchmarks for unsupervised instance segmentation and object detection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。