提出NCGR方法,让摄像头外参漂移时3D检测仍准
NCGR: Noise-Conditional Gated Rectification for Camera Extrinsic Perturbations in BEV 3D Object Detection
- 用噪声条件门控校正投影偏移,无需估计完整外参
- 动态扰动下NDS达39.69%,比BEVFormer高11.69个百分点
- 训练时自动生成条件,推理无需外参信息,适合实际部署
基于摄像头的鸟瞰图(BEV)3D检测通常假设相机外参准确且固定。在使用空间交叉注意力(SCA)的检测器中,外参扰动会改变BEV参考点在图像平面上的投影位置,导致查询采样错误区域特征,降低检测性能。为解决此问题,提出噪声条件门控校正(NCGR),在不显式估计六自由度外参修正的情况下补偿投影误差。对每个查询-相机对,预测二维校正偏移并由相机级门控调制,在原始可变形采样前修正基投影。训练时,通过调度插值将扰动导出量替换为从相机特征预测的辅助标量生成的对应量,实现无扰动元数据的盲推理。训练采用共享权重的干净教师/扰动学生对,校正模块由两分支间的BEV一致性目标监督。在nuScenes上评估模拟动态和静态外参扰动,五相机动态压力测试中,NCGR达到39.69% NDS,优于BEVFormer的28.00%和CAPE的33.23%。在无扰动外参下,性能与BEVFormer相当。
原文摘要 · Abstract (English)
Camera-based bird's-eye-view (BEV) 3D detection typically assumes accurate and fixed camera extrinsics. In detectors using spatial cross-attention (SCA), extrinsic perturbations displace the image-plane projections of BEV reference points, causing queries to sample features from incorrect regions and degrading detection performance. To address this failure mode, Noise-Conditional Gated Rectification (NCGR) is proposed to compensate for projection errors without explicitly estimating a full six-degree-of-freedom extrinsic correction. For each query-camera pair, a 2D rectification offset is predicted and modulated by a camera-level gate to rectify the base projection before native deformable sampling. During training, the perturbation-derived quantities used to construct the condition and gate are gradually replaced through scheduled interpolation by counterparts generated from an auxiliary scalar predicted from camera features. This transition enables blind inference without perturbation metadata. During training, a weight-shared clean-teacher/perturbed-student pair is used, and the rectification module is supervised by a BEV-consistency objective between the two branches. NCGR is evaluated on nuScenes with simulated dynamic and static extrinsic perturbations. In a five-camera dynamic stress test, NCGR achieves 39.69% NDS, compared with 28.00% for BEVFormer and 33.23% for CAPE. Under clean extrinsics, NCGR maintains performance comparable to that of BEVFormer.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。