针对鱼眼相机畸变,提出无需投影的3D目标检测新方法。
Distortion-Aware PETR for BEV Object Detection with Mixed Pinhole-Fisheye Cameras

- 设计自适应位置编码与双向特征调制模块,适配鱼眼几何
- 在KITTI-360上优于基准模型,性能显著提升
- 揭示学习适配与显式重参数化存在冲突,具指导意义
鱼眼相机因低成本和全景视场被广泛用于自动驾驶感知,但其严重径向畸变破坏了鸟瞰图(BEV)检测器的均匀采样假设。为此,本文提出无投影的畸变感知PETR(DAPETR),专为混合针孔-鱼眼相机配置设计。DAPETR引入两个可学习的自适应模块:统一的畸变感知位置嵌入,协调图像表征与鱼眼几何的位置编码;双向特征-几何协同调制模块,实现图像特征与3D位置嵌入的相互适应。在转换后的KITTI-360基准上,系统对比DAPETR与极坐标下的PETR(PolarPETR)。结果表明,尽管两者均优于基线,但所提学习模块表现更优;关键发现是两种策略结合时出现负向交互,说明学习适配与显式几何重参数化可能冲突。最终模型显著推进了鱼眼相机下BEV检测的研究与基准,为非图像校正的畸变感知3D感知设计提供关键洞见。
原文摘要 · Abstract (English)
Fisheye cameras are widely deployed in autonomous driving perception suites for their low cost and full-coverage field of view (FOV), yet their potential remains underleveraged in 3D object detection. Severe radial distortion challenges most BEV detectors by violating the fundamental assumption of uniform sampling. To bridge this gap, we propose Distortion-Aware PETR (DAPETR), a projection-free detector tailored for mixed pinhole-fisheye camera setups. DAPETR incorporates two key learned-adaptive modules: a unified distortion-aware positional embedding that harmonizes positional encodings for image representations with fisheye geometry, and a bidirectional feature-geometry co-modulation module that mutually adapts image features and 3D positional embeddings. In our experiments on a converted KITTI-360 benchmark, we systematically compare our learned adaptive approach against PETR in polar coordinates (PolarPETR). We find that while both methods improve over the baseline, our learned modules achieve superior performance. Crucially, we uncover a negative interaction when combining both strategies, revealing that learned adaptation and explicit geometric reparameterization can conflict. Our final DAPETR model significantly advances the research and benchmark for fisheye BEV detection, providing critical insights into effective distortion-aware 3D perception design other than image rectification.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。