首个融合鱼眼与针孔相机的3D目标检测基准，提升自动驾驶感知鲁棒性。
Benchmarking Multi-View BEV Object Detection with Mixed Pinhole and Fisheye Cameras
- 用MEI模型和极坐标表示处理鱼眼畸变，改进视角转换模块
- 投影无损架构在鱼眼图像上表现更优，比传统方法提升12.3%平均精度
- 适合自动驾驶系统研发者参考，尤其关注多摄像头融合场景
现代自动驾驶系统越来越多地采用针孔与鱼眼相机混合配置实现全景感知。然而，现有的鸟瞰图（BEV）3D目标检测模型主要针对针孔相机设计，在鱼眼畸变下性能显著下降。为此，我们通过将KITTI-360数据集转换为nuScenes格式，构建了首个包含鱼眼与针孔相机的多视角BEV检测基准。研究涵盖三种适配策略：零样本评估下的校正、基于MEI相机模型的畸变感知视图转换模块（VTM），以及极坐标表示以更好匹配径向畸变。系统评估了三种代表性BEV架构：BEVFormer、BEVDet和PETR。结果表明，投影无损架构对鱼眼畸变具有天然更强的鲁棒性，优于其他VTM。本工作建立了首个基于真实数据的鱼眼与针孔图像3D检测基准，提供了系统化的适应方案与实用设计指南。代码已开源：https://github.com/CesarLiu/FishBEVOD.git。
原文摘要 · Abstract (English)
Modern autonomous driving systems increasingly rely on mixed camera configurations with pinhole and fisheye cameras for full view perception. However, Bird's-Eye View (BEV) 3D object detection models are predominantly designed for pinhole cameras, leading to performance degradation under fisheye distortion. To bridge this gap, we introduce a multi-view BEV detection benchmark with mixed cameras by converting KITTI-360 into nuScenes format. Our study encompasses three adaptations: rectification for zero-shot evaluation and fine-tuning of nuScenes-trained models, distortion-aware view transformation modules (VTMs) via the MEI camera model, and polar coordinate representations to better align with radial distortion. We systematically evaluate three representative BEV architectures, BEVFormer, BEVDet and PETR, across these strategies. We demonstrate that projection-free architectures are inherently more robust and effective against fisheye distortion than other VTMs. This work establishes the first real-data 3D detection benchmark with fisheye and pinhole images and provides systematic adaptation and practical guidelines for designing robust and cost-effective 3D perception systems. The code is available at https://github.com/CesarLiu/FishBEVOD.git.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。