arXiv:2503.21449cs.CV2025-03TPAMI被引 4

无需投影或分步模型,直接生成逼真3D语义场景数据。

Towards Generating Realistic 3D Semantic Training Data for Autonomous Driving

  • 直接在3D空间生成带语义标注的场景点云,避免图像投影误差。
  • 用合成数据训练分割网络,性能比纯真实数据提升1.2% mIoU。
  • 适合自动驾驶数据集扩展,大幅减少人工标注成本。

语义场景理解对机器人和计算机视觉至关重要,在自动驾驶中,3D语义分割是实现安全导航的关键。尽管该领域进展显著,但3D数据的采集与标注仍存在瓶颈。为克服这一限制,已有研究使用合成数据按需生成标注数据,但真实数据与仿真数据间仍存在域差距。近期扩散模型兴起,可生成接近真实的合成数据。这些生成模型被用于3D场景级数据生成并附带语义标注,但现有方法多依赖图像投影或分阶段、多分辨率的解耦模型,中间转换过程引入误差,影响生成质量。本文提出一种新方法,无需任何投影或分阶段训练,直接生成高质量3D语义场景数据,优于当前最先进方法。我们还系统评估了合成数据作为标注数据训练语义分割网络的效果:在真实标签基础上加入本方法生成的数据,使分割模型性能提升1.2% mIoU。结果表明,生成的场景级点云可用于扩充现有数据集,降低标注负担。代码已开源:https://github.com/PRBonn/3DiSS。

原文摘要 · Abstract (English)

Semantic scene understanding is crucial for robotics and computer vision applications. In autonomous driving, 3D semantic segmentation plays an important role for enabling safe navigation. Despite significant advances in the field, the complexity of collecting and annotating 3D data is a bottleneck in this developments. To overcome that data annotation limitation, synthetic simulated data has been used to generate annotated data on demand. There is still, however, a domain gap between real and simulated data. More recently, diffusion models have been in the spotlight, enabling close-to-real data synthesis. Those generative models have been recently applied to the 3D data domain for generating scene-scale data with semantic annotations. Still, those methods either rely on image projection or decoupled models trained with different resolutions in a coarse-to-fine manner. Such intermediary representations impact the generated data quality due to errors added in those transformations. In this work, we propose a novel approach able to generate 3D semantic scene-scale data without relying on any projection or decoupled trained multi-resolution models, achieving more realistic semantic scene data generation compared to previous state-of-the-art methods. Besides improving 3D semantic scene-scale data synthesis, we thoroughly evaluate the use of the synthetic scene samples as labeled data to train a semantic segmentation network. In our experiments, we show that using the synthetic annotated data generated by our method as training data together with the real semantic segmentation labels, leads to an improvement in the semantic segmentation model performance. Our results show the potential of generated scene-scale point clouds to generate more training data to extend existing datasets, reducing the data annotation effort. Our code is available at https://github.com/PRBonn/3DiSS.

3D生成语义分割自动驾驶扩散模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。