arXiv:2508.12695cs.CV2025-08中稿 · IEEE Intelligent V…被引 3

用神经渲染技术适配不同传感器,解决自动驾驶3D检测的跨传感器性能下降问题。

Neural Rendering for Sensor Adaptation in 3D Object Detection

  • 基于神经渲染构建数据转换管道,实现传感器配置的自动适配。
  • 在CamShift数据集上验证了跨传感器性能下降超40%,且BEVFormer最鲁棒。
  • 无需重新采集数据,可高效复用训练集,适合多车型自动驾驶系统部署。

自动驾驶车辆因车型差异导致摄像头传感器配置各异,使模型在新传感器上性能显著下降。本文提出CamShift数据集(基于CARLA模拟nuScenes场景),专门用于研究紧凑型车与SUV间的跨传感器域差距。实验表明,现有3D检测器在跨传感器条件下平均性能下降超过40%;其中基于密集鸟瞰图(BEV)和反向投影的模型(如BEVFormer)表现更稳健。为此,本文设计了一种基于神经渲染的数据驱动传感器适配方案,可将完整数据集转换为任意传感器配置。该方法在所有测试模型上均提升性能,显著缩小域差距,并大幅减少对新数据采集的依赖,实现跨车型数据高效复用。相关数据集与基准测试已开源。

原文摘要 · Abstract (English)

Autonomous vehicles often have varying camera sensor setups, which is inevitable due to restricted placement options for different vehicle types. Training a perception model on one particular setup and evaluating it on a new, different sensor setup reveals the so-called cross-sensor domain gap, typically leading to a degradation in accuracy. In this paper, we investigate the impact of the cross-sensor domain gap on state-of-the-art 3D object detectors. To this end, we introduce CamShift, a dataset inspired by nuScenes and created in CARLA to specifically simulate the domain gap between subcompact vehicles and sport utility vehicles (SUVs). Using CamShift, we demonstrate significant cross-sensor performance degradation, identify robustness dependencies on model architecture, and propose a data-driven solution to mitigate the effect. On the one hand, we show that model architectures based on a dense Bird's Eye View (BEV) representation with backward projection, such as BEVFormer, are the most robust against varying sensor configurations. On the other hand, we propose a novel data-driven sensor adaptation pipeline based on neural rendering, which can transform entire datasets to match different camera sensor setups. Applying this approach improves performance across all investigated 3D object detectors, mitigating the cross-sensor domain gap by a large margin and reducing the need for new data collection by enabling efficient data reusability across vehicles with different sensor setups. The CamShift dataset and the sensor adaptation benchmark are available at https://dmholtz.github.io/camshift/.

3D检测神经渲染传感器适配自动驾驶

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。