用生成数据提升自动驾驶鸟瞰图语义分割鲁棒性,解决合成数据噪声问题。
NRSeg: Noise-Resilient Learning for BEV Semantic Segmentation via Driving World Models

- 通过视角几何一致性度量评估合成数据对模型的指导价值
- 在无监督和半监督场景下分别提升13.8%和11.4%的mIoU
- 适合关注自动驾驶感知鲁棒性的研究者与工程师
鸟瞰图(BEV)语义分割是端到端自动驾驶系统中不可或缺的感知任务。无监督和半监督学习在真实场景应用中至关重要,但受限于标注数据分布单一而表现不佳。本文探索利用驾驶世界模型生成的合成数据,提升标注数据多样性以增强BEV分割的鲁棒性。然而初步发现,合成数据中的生成噪声会损害模型学习效率。为此,本文提出NRSeg——一种面向BEV语义分割的抗噪声学习框架。首先设计了视角-几何一致性度量(PGCM),通过比较生成数据的透视道路掩码与从BEV标签投影得到的掩码之间的对齐程度,定量评估合成数据的指导能力。其次提出双分布并行预测(BiDPP),通过同时预测多项分布和狄利克雷分布来增强模型鲁棒性:前者高效输出语义概率,后者采用证据深度学习实现不确定性量化。此外,设计分层局部语义排除(HLSE)模块,解决BEV语义分割中语义类别非互斥的固有问题。实验表明,NRSeg在无监督和半监督任务中分别取得最高13.8%和11.4%的mIoU提升,达到当前最优性能。代码将公开于https://github.com/lynn-yu/NRSeg。
原文摘要 · Abstract (English)
Birds' Eye View (BEV) semantic segmentation is an indispensable perception task in end-to-end autonomous driving systems. Unsupervised and semi-supervised learning for BEV tasks, as pivotal for real-world applications, underperform due to the homogeneous distribution of the labeled data. In this work, we explore the potential of synthetic data from driving world models to enhance the diversity of labeled data for robustifying BEV segmentation. Yet, our preliminary findings reveal that generation noise in synthetic data compromises efficient BEV model learning. To fully harness the potential of synthetic data from world models, this paper proposes NRSeg, a noise-resilient learning framework for BEV semantic segmentation. Specifically, a Perspective-Geometry Consistency Metric (PGCM) is proposed to quantitatively evaluate the guidance capability of generated data for model learning. This metric originates from the alignment measure between the perspective road mask of generated data and the mask projected from the BEV labels. Moreover, a Bi-Distribution Parallel Prediction (BiDPP) is designed to enhance the inherent robustness of the model, where the learning process is constrained through parallel prediction of multinomial and Dirichlet distributions. The former efficiently predicts semantic probabilities, whereas the latter adopts evidential deep learning to realize uncertainty quantification. Furthermore, a Hierarchical Local Semantic Exclusion (HLSE) module is designed to address the non-mutual exclusivity inherent in BEV semantic segmentation tasks. Experimental results demonstrate that NRSeg achieves state-of-the-art performance, yielding the highest improvements in mIoU of 13.8% and 11.4% in unsupervised and semi-supervised BEV segmentation tasks, respectively. The source code will be made publicly available at https://github.com/lynn-yu/NRSeg.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。