用表面法线图提升事件相机的物体检测精度
Enhancing Event-based Object Detection with Monocular Normal Maps
- 融合法线图、图像和事件数据，利用几何先验增强检测
- 在复杂光照下提升3.0%的检测准确率，优于双模态基线
- 适合自动驾驶中弱光或反光场景下的视觉系统优化
自动驾驶中的物体检测常受复杂光照影响。事件相机虽具鲁棒性，但对反射等突变对比度敏感，易产生密集误触发信号。为此，本文利用由RGB生成的表面法线图作为显式几何约束。即使在RGB质量下降时，法线图仍保留低频结构先验，有效辅助事件相机检测。我们提出NRE-Net，一个三模态框架，融合法线图提供的结构先验、RGB提供的外观上下文与事件数据捕捉的高频动态信息。自适应双流融合模块（ADFM）首先对齐几何与外观线索，随后事件感知融合模块（EAFM）选择性整合事件动态。在DSEC-Det-sub与PKU-DAVIS-SOD数据集上的大量实验表明，引入几何先验使检测性能比双模态基线额外提升3.0% AP50；相比SFNet（+2.7%）和SODFormer（+7.1%），本方法表现更优。
原文摘要 · Abstract (English)
Object detection in autonomous driving is frequently compromised by complex illumination. While event cameras offer a robust solution, they are susceptible to sudden contrast changes such as reflections which often trigger dense, misleading event signals. To overcome this, we leverage RGB-derived surface normal maps as explicit geometric constraints. Crucially, even when RGB degrades, they preserve low-frequency structural priors that effectively assist in event-based detection. Consequently, we present NRE-Net, a trimodal framework that integrates structural priors from surface Normal maps, appearance context from RGB images, and high-frequency dynamics from Events. The Adaptive Dual-stream Fusion Module (ADFM) first aligns geometric and appearance cues, followed by the Event-modality Aware Fusion Module (EAFM) which selectively integrates event dynamics. Extensive evaluations on DSEC-Det-sub and PKU-DAVIS-SOD demonstrate that incorporating geometric priors yields an additional 3.0% AP50 gain over dual-modal baselines, while our approach consistently outperforms fusion methods such as SFNet (+2.7%) and SODFormer (+7.1%).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。