提出端到端可微校准的雷达相机融合检测方法，提升水面环境感知精度。
calibfusion: Transformer-Based Differentiable Calibration for Radar-Camera Fusion Detection in Water-Surface Environments
- 基于Transformer的隐式外参优化，联合检测任务训练校准参数。
- 在WaterScenes和FLOW数据集上提升2D检测性能，对错配具有鲁棒性。
- 适用于水面、无纹理场景，也适用于道路等复杂环境。
毫米波雷达与相机融合能提升恶劣光照与天气下的感知能力，但其性能对雷达-相机外参校准敏感：残余偏移会扭曲雷达投影至图像平面，影响跨模态特征融合。现有校准方法多针对道路与城市场景设计，依赖丰富结构和物体约束，而水面环境具有大范围无纹理区域、稀疏间断目标及由波浪/镜面反射引起的雷达杂波，削弱了以物体为中心的匹配能力。本文提出CalibFusion，一种校准条件化的雷达-相机融合检测器，通过检测目标端到端学习隐式外参优化。该方法构建多帧时序感知的雷达密度表示，结合强度加权与多普勒引导的快速变化杂波抑制；跨模态Transformer模块预测置信度门控的初始外参修正，并通过可微投影-撒点算子融合生成校准条件化图像平面雷达特征。在WaterScenes和FLOW数据集上的实验表明，融合检测性能提升，且在合成错配下表现鲁棒，敏感性分析与雷达-图像叠加可视化支持有效性。nuScenes上的结果表明，该优化机制可泛化至非水面场景。
原文摘要 · Abstract (English)
Millimeter-wave (mmWave) Radar--Camera fusion improves perception under adverse illumination and weather, but its performance is sensitive to Radar--Camera extrinsic calibration: residual misalignment biases Radar-to-image projection and degrades cross-modal aggregation for downstream 2D detection. Existing calibration and auto-calibration methods are mainly developed for road and urban scenes with abundant structures and object constraints, whereas water-surface environments feature large textureless regions, sparse and intermittent targets, and wave-/specular-induced Radar clutter, which weakens explicit object-centric matching. We propose CalibFusion, a calibration-conditioned Radar--Camera fusion detector that learns implicit extrinsic refinement end-to-end with the detection objective. CalibFusion builds a multi-frame persistence-aware Radar density representation with intensity weighting and Doppler-guided suppression of fast-varying clutter. A cross-modal transformer interaction module predicts a confidence-gated refinement of the initial extrinsics, which is integrated through a differentiable projection-and-splatting operator to generate calibration-conditioned image-plane Radar features. Experiments on WaterScenes and FLOW show improved fusion-based 2D detection and robustness under synthetic miscalibration, supported by sensitivity analyses and qualitative Radar-to-image overlays. Results on nuScenes indicate that the refinement mechanism transfers beyond water-surface scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。