轻量级模型一次前向计算完成自动驾驶多任务感知,兼顾精度与效率。
Towards Compact Autonomous Driving Perception with Balanced Learning and Multi-sensor Fusion

- 设计紧凑的多任务学习模型,统一处理语义分割、深度估计等任务。
- 在CARLA和nuScenes数据集上参数少但性能更优,推理更快且显存占用低。
- 支持多传感器融合,适用于真实场景与仿真环境,适合部署于车载系统。
我们提出一种新型紧凑型深度多任务学习模型,可在一次前向传播中同时完成多种自动驾驶感知任务,包括多视角语义分割、深度估计、激光雷达分割及鸟瞰图投影,无需依赖其他模型。为应对多任务带来的学习不平衡问题,引入自适应损失权重算法。通过数据预处理与中间层传感器融合技术,模型可整合来自RGB相机、动态视觉传感器(DVS)和多个位置激光雷达的多模态输入,实现对动态环境的更好理解。消融实验表明,采用该方法训练的模型变体性能更优;对比实验显示,其在保持显著更少参数的同时,仍优于若干近期模型组合。该模型在3个CARLA仿真数据集和1个真实世界nuScenes-lidarseg数据集上表现一致。代码与相关文件已公开:https://github.com/oskarnatan/compact-perception。
原文摘要 · Abstract (English)
We present a novel compact deep multi-task learning model to handle various autonomous driving perception tasks in one forward pass. The model performs multiple views of semantic segmentation, depth estimation, light detection and ranging (LiDAR) segmentation, and bird's eye view projection simultaneously without being supported by other models. We also provide an adaptive loss weighting algorithm to tackle the imbalanced learning issue that occurred due to plenty of given tasks. Through data pre-processing and intermediate sensor fusion techniques, the model can process and combine multiple input modalities retrieved from RGB cameras, dynamic vision sensors (DVS), and LiDAR placed at several positions on the ego vehicle. Therefore, a better understanding of a dynamically changing environment can be achieved. Based on the ablation study, the model variant trained with our proposed method achieves a better performance. Furthermore, a comparative study is also conducted to clarify its performance and effectiveness against the combination of some recent models. As a result, our model maintains better performance even with much fewer parameters. Hence, the model can inference faster with less GPU memory utilization. Moreover, the result tends to be consistent in 3 different CARLA simulation datasets and 1 real-world nuScenes-lidarseg dataset. To support future research, we share codes and other files publicly at https://github.com/oskarnatan/compact-perception.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。