用双目视觉提升3D检测精度，同时实现实时推理。
StereoDETR: Stereo-based Transformer for 3D Object Detection
- 双分支结构融合单目与双目特征，通过可微深度采样耦合。
- 在KITTI数据集上超越单目方法速度，行人和骑行者检测达新高。
- 无需额外标注即可处理遮挡，适合自动驾驶实时感知场景。
相比单目3D目标检测，基于双目的方法虽显著提升精度，但计算开销大、延迟高。当前最先进双目方法精度是单目方法的两倍，但推理速度仅为一半。本文提出StereoDETR，一种基于DETR的高效双目3D检测框架。该框架包含单目DETR分支和双目分支：单目分支在2D DETR基础上增加通道以预测目标尺度、方向和采样点；双目分支利用低成本多尺度视差特征预测物体级深度图。两分支仅通过可微深度采样策略耦合。为应对遮挡，引入无需额外标注的约束监督策略。StereoDETR实现实时推理，是首个在速度上超越单目方法的双目检测方法。在公开KITTI基准上达到竞争性精度，行人与骑行者子集创下新纪录。代码已开源：https://github.com/shiyi-mu/StereoDETR-OPEN。
原文摘要 · Abstract (English)
Compared to monocular 3D object detection, stereo-based 3D methods offer significantly higher accuracy but still suffer from high computational overhead and latency. The state-of-the-art stereo 3D detection method achieves twice the accuracy of monocular approaches, yet its inference speed is only half as fast. In this paper, we propose StereoDETR, an efficient stereo 3D object detection framework based on DETR. StereoDETR consists of two branches: a monocular DETR branch and a stereo branch. The DETR branch is built upon 2D DETR with additional channels for predicting object scale, orientation, and sampling points. The stereo branch leverages low-cost multi-scale disparity features to predict object-level depth maps. These two branches are coupled solely through a differentiable depth sampling strategy. To handle occlusion, we introduce a constrained supervision strategy for sampling points without requiring extra annotations. StereoDETR achieves real-time inference and is the first stereo-based method to surpass monocular approaches in speed. It also achieves competitive accuracy on the public KITTI benchmark, setting new state-of-the-art results on pedestrian and cyclist subsets. The code is available at https://github.com/shiyi-mu/StereoDETR-OPEN.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。