用多视角图像引导,分层提升工业场景点云分割精度。
Hierarchical Image-Guided 3D Point Cloud Segmentation in Industrial Scenes via Multi-View Bayesian Fusion
- 先用图像生成实例级掩码,再细化到零件级分割
- 多视角融合后通过贝叶斯更新确保语义一致
- 适合需要少标注且抗遮挡的工业三维场景
可靠的3D分割对理解密集布局和多尺度物体的工业场景至关重要。此类场景中,严重遮挡削弱了物体间的几何边界,而物体尺度差异大导致端到端模型难以同时捕捉粗粒度与细粒度特征。现有基于点云的方法需昂贵标注,图像引导方法则常存在跨视图语义不一致问题。为此,我们提出一种分层图像引导3D分割框架,逐步从实例级精化至零件级。实例分割通过渲染俯视图,将YOLO-World提示的SAM生成掩码投影回3D点云;零件级分割则对前一阶段获得的每个实例,渲染多视角图像,重复2D分割与反投影流程,并通过贝叶斯更新融合各视角结果以保证语义一致性。在真实工厂数据上的实验表明,该方法有效应对遮挡与结构复杂性,实现高且稳定的类别级mIoU。公共数据集上的额外评估验证了框架的泛化能力,凸显其鲁棒性、标注效率及对多样化3D环境的适应性。
原文摘要 · Abstract (English)
Reliable 3D segmentation is critical for understanding complex scenes with dense layouts and multi-scale objects, as commonly seen in industrial environments. In such scenarios, heavy occlusion weakens geometric boundaries between objects, and large differences in object scale will cause end-to-end models fail to capture both coarse and fine details accurately. Existing 3D point-based methods require costly annotations, while image-guided methods often suffer from semantic inconsistencies across views. To address these challenges, we propose a hierarchical image-guided 3D segmentation framework that progressively refines segmentation from instance-level to part-level. Instance segmentation involves rendering a top-view image and projecting SAM-generated masks prompted by YOLO-World back onto the 3D point cloud. Part-level segmentation is subsequently performed by rendering multi-view images of each instance obtained from the previous stage and applying the same 2D segmentation and back-projection process at each view, followed by Bayesian updating fusion to ensure semantic consistency across views. Experiments on real-world factory data demonstrate that our method effectively handles occlusion and structural complexity, achieving consistently high per-class mIoU scores. Additional evaluations on public dataset confirm the generalization ability of our framework, highlighting its robustness, annotation efficiency, and adaptability to diverse 3D environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。