首个实时立体3D目标检测框架,专为动态感知优化。
Real-time Stereo-based 3D Object Detection for Streaming Perception
- 基于历史信息预测下一时刻3D目标,端到端设计
- 在KITTI上提升流感知平均精度4.33%(最高)
- 适合自动驾驶实时感知系统研发者参考
自主驾驶的感知系统需快速响应环境变化。近期提出的新任务“流感知”将延迟与精度统一为单一指标,用于视频在线感知评估。本文提出首个面向流感知的实时立体3D目标检测框架StreamDSGN。该框架端到端设计,通过利用历史信息直接预测下一时刻物体的3D属性,缓解了流感知中的精度下降问题。进一步引入三项策略:(1) 基于特征流的融合方法,在当前时刻生成伪下一帧特征,解决特征与真实标签间的错位问题;(2) 额外回归损失,显式监督连续帧间物体运动的一致性;(3) 采用大核主干网络,扩大感受野以有效捕捉因物体位置变化引发的长程空间上下文特征。在KITTI Tracking数据集上的实验表明,相比强基线模型,StreamDSGN将流感知平均精度最高提升4.33%。代码已开源:https://github.com/weiyangdaren/streamDSGN-pytorch。
原文摘要 · Abstract (English)
The ability to promptly respond to environmental changes is crucial for the perception system of autonomous driving. Recently, a new task called streaming perception was proposed. It jointly evaluate the latency and accuracy into a single metric for video online perception. In this work, we introduce StreamDSGN, the first real-time stereo-based 3D object detection framework designed for streaming perception. StreamDSGN is an end-to-end framework that directly predicts the 3D properties of objects in the next moment by leveraging historical information, thereby alleviating the accuracy degradation of streaming perception. Further, StreamDSGN applies three strategies to enhance the perception accuracy: (1) A feature-flow-based fusion method, which generates a pseudo-next feature at the current moment to address the misalignment issue between feature and ground truth. (2) An extra regression loss for explicit supervision of object motion consistency in consecutive frames. (3) A large kernel backbone with a large receptive field for effectively capturing long-range spatial contextual features caused by changes in object positions. Experiments on the KITTI Tracking dataset show that, compared with the strong baseline, StreamDSGN significantly improves the streaming average precision by up to 4.33%. Our code is available at https://github.com/weiyangdaren/streamDSGN-pytorch.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。