arXiv:2504.03129cs.ROcs.AI2025-04被引 2

用少量2D图像生成一致的3D物体分割,提升机器人操作性能。

GraphSeg: Segmented 3D Representations via Graph Edge Addition and Contraction

  • 通过图边添加与收缩,融合2D相似性和推断3D结构
  • 仅需少量图像即达更高精度,优于已有方法
  • 适合机器人场景下的3D语义分割任务

在非结构化环境中运行的机器人通常需要准确且一致的物体级表征。这通常要求将单个物体从周围环境中分割出来。尽管最近的大模型如Segment Anything(SAM)在2D图像分割上表现优异,但这些进展难以直接应用于真实3D世界,常导致过分割且跨视角掩码对应不一致。本文提出GraphSeg框架,仅需一组稀疏的环境2D图像(无需深度信息),即可生成一致的3D物体分割结果。GraphSeg通过添加图边构建双重对应图:一个基于2D像素相似性,另一个基于推断的3D结构。将分割问题建模为边添加后图收缩的过程,从而将多个2D掩码合并为统一的物体级分割。随后可利用3D基础模型生成分段3D表示。实验表明,GraphSeg在桌面上场景中达到当前最优性能,并显著提升下游机器人操控任务的表现。代码已公开于https://github.com/tomtang502/graphseg.git。

原文摘要 · Abstract (English)

Robots operating in unstructured environments often require accurate and consistent object-level representations. This typically requires segmenting individual objects from the robot's surroundings. While recent large models such as Segment Anything (SAM) offer strong performance in 2D image segmentation. These advances do not translate directly to performance in the physical 3D world, where they often over-segment objects and fail to produce consistent mask correspondences across views. In this paper, we present GraphSeg, a framework for generating consistent 3D object segmentations from a sparse set of 2D images of the environment without any depth information. GraphSeg adds edges to graphs and constructs dual correspondence graphs: one from 2D pixel-level similarities and one from inferred 3D structure. We formulate segmentation as a problem of edge addition, then subsequent graph contraction, which merges multiple 2D masks into unified object-level segmentations. We can then leverage \emph{3D foundation models} to produce segmented 3D representations. GraphSeg achieves robust segmentation with significantly fewer images and greater accuracy than prior methods. We demonstrate state-of-the-art performance on tabletop scenes and show that GraphSeg enables improved performance on downstream robotic manipulation tasks. Code available at https://github.com/tomtang502/graphseg.git.

3D分割机器人感知图神经网络无深度感知

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。