通过多尺度图像特征生成多尺度鸟瞰图特征,提升不同视角下行人检测精度。
MSMVD: Exploiting Multi-scale Image Features via Multi-scale BEV Features for Multi-view Pedestrian Detection
- 将多视角图像的多尺度特征逐级投影到鸟瞰图空间生成多尺度BEV特征
- 在GMVD数据集上比此前最优方法提升4.5个MODA点
- 适合处理视角间尺度差异大或目标本身尺度极端的行人检测场景
多视角行人检测(MVPD)旨在从多视角图像中以鸟瞰图(BEV)形式检测行人。尽管端到端可训练的深度学习方法已取得显著进展,但在检测视角中尺度一致过小或过大,或不同视角间尺度差异极大的行人时仍表现不佳,原因在于未有效利用多尺度图像特征生成BEV特征。为此,本文提出一种新方法——多尺度多视角检测(MSMVD)。MSMVD通过将各视角提取的多尺度图像特征逐尺度投影至BEV空间,生成多尺度BEV特征,每个特征继承对应尺度图像特征的特性,从而实现对尺度一致偏小或偏大行人的精确检测。随后,采用特征金字塔网络融合多视图多尺度BEV特征,提升跨视角尺度差异大的行人检测性能。大量实验表明,通过多尺度图像特征生成多尺度BEV特征能显著提升检测效果,MSMVD在GMVD数据集上比之前最优方法高出4.5个MODA点。
原文摘要 · Abstract (English)
Multi-View Pedestrian Detection (MVPD) aims to detect pedestrians in the form of a bird's eye view (BEV) from multi-view images. In MVPD, end-to-end trainable deep learning methods have progressed greatly. However, they often struggle to detect pedestrians with consistently small or large scales in views or with vastly different scales between views. This is because they do not exploit multi-scale image features to generate the BEV feature and detect pedestrians. To overcome this problem, we propose a novel MVPD method, called Multi-Scale Multi-View Detection (MSMVD). MSMVD generates multi-scale BEV features by projecting multi-scale image features extracted from individual views into the BEV space, scale-by-scale. Each of these BEV features inherits the properties of its corresponding scale image features from multiple views. Therefore, these BEV features help the precise detection of pedestrians with consistently small or large scales in views. Then, MSMVD combines information at different scales of multiple views by processing the multi-scale BEV features using a feature pyramid network. This improves the detection of pedestrians with vastly different scales between views. Extensive experiments demonstrate that exploiting multi-scale image features via multi-scale BEV features greatly improves the detection performance, and MSMVD outperforms the previous highest MODA by $4.5$ points on the GMVD dataset.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。