用3D生成数据预训练,提升点云实例分割性能。
Pre-training with 3D Synthetic Data: Learning 3D Point Cloud Instance Segmentation from 3D Synthetic Scenes
- 用Point-E生成3D场景点云数据进行预训练。
- 在ScanNet数据集上比基线模型提升4.2%的mAP。
- 适合做3D点云分割但缺乏标注数据的研究者。
近年来,3D点云数据在机器人、自动驾驶等实际应用中日益受到重视,因其能提供真实尺寸与空间信息。然而,构建高质量3D点云数据集成本高昂,需对大规模3D空间中的每个点进行类别和实例标注。为缓解这一问题,本文提出一种基于3D生成模型的预训练方法,利用Point-E生成3D点云数据,构建合成3D场景用于训练点云实例分割模型。尽管近期出现更先进的3D生成模型,但即使使用早期的Point-E也已能有效支持该预训练策略。实验表明,该方法在ScanNet数据集上显著优于基线模型,验证了3D生成数据在点云实例分割中的有效性。
原文摘要 · Abstract (English)
In the recent years, the research community has witnessed growing use of 3D point cloud data for the high applicability in various real-world applications. By means of 3D point cloud, this modality enables to consider the actual size and spatial understanding. The applied fields include mechanical control of robots, vehicles, or other real-world systems. Along this line, we would like to improve 3D point cloud instance segmentation which has emerged as a particularly promising approach for these applications. However, the creation of 3D point cloud datasets entails enormous costs compared to 2D image datasets. To train a model of 3D point cloud instance segmentation, it is necessary not only to assign categories but also to provide detailed annotations for each point in the large-scale 3D space. Meanwhile, the increase of recent proposals for generative models in 3D domain has spurred proposals for using a generative model to create 3D point cloud data. In this work, we propose a pre-training with 3D synthetic data to train a 3D point cloud instance segmentation model based on generative model for 3D scenes represented by point cloud data. We directly generate 3D point cloud data with Point-E for inserting a generated data into a 3D scene. More recently in 2025, although there are other accurate 3D generation models, even using the Point-E as an early 3D generative model can effectively support the pre-training with 3D synthetic data. In the experimental section, we compare our pre-training method with baseline methods indicated improved performance, demonstrating the efficacy of 3D generative models for 3D point cloud instance segmentation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。