解决单目3D检测中2D匹配误判3D预测的问题
Mono3DV: Monocular 3D Object Detection with 3D-Aware Bipartite Matching and Variational Query DeNoising
- 引入3D感知的二分匹配,直接用3D几何信息计算匹配代价
- 在KITTI上达到新最好效果,无需外部数据
- 提出变分查询去噪机制,缓解传统去噪梯度消失问题
尽管基于DETR的架构在单目3D目标检测中展现出巨大潜力,但其常受制于一个关键局限:二分匹配过程排除了3D属性。这源于从单目图像进行3D估计的固有病态性,导致训练中不稳定性。因此,高质量的3D预测可能被仅基于2D的匹配标准错误抑制,造成次优结果。为此,我们提出Mono3DV,一种新型Transformer框架。其引入三项关键创新:首先,设计3D感知的二分匹配策略,将3D几何信息直接纳入匹配代价,解决纯2D标准引起的错配;其次,为稳定匹配过程,提出3D去噪方案以应对3D属性融合带来的不稳定性;最后,针对传统去噪技术存在的梯度消失问题,提出新颖的变分查询去噪机制,显著提升模型性能。该方法未使用任何外部数据,在KITTI 3D目标检测基准上取得当前最优结果。
原文摘要 · Abstract (English)
While DETR-like architectures have demonstrated significant potential for monocular 3D object detection, they are often hindered by a critical limitation: the exclusion of 3D attributes from the bipartite matching process. This exclusion arises from the inherent ill-posed nature of 3D estimation from monocular image, which introduces instability during training. Consequently, high-quality 3D predictions can be erroneously suppressed by 2D-only matching criteria, leading to suboptimal results. To address this, we propose Mono3DV, a novel Transformer-based framework. Our approach introduces three key innovations. First, we develop a 3D-Aware Bipartite Matching strategy that directly incorporates 3D geometric information into the matching cost, resolving the misalignment caused by purely 2D criteria. Second, it is important to stabilize the Bipartite Matching to resolve the instability occurring when integrating 3D attributes. Therefore, we propose 3D-DeNoising scheme in the training phase. Finally, recognizing the gradient vanishing issue associated with conventional denoising techniques, we propose a novel Variational Query DeNoising mechanism to overcome this limitation, which significantly enhances model performance. Without leveraging any external data, our method achieves state-of-the-art results on the KITTI 3D object detection benchmark.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。