用事件相机提升自动驾驶感知，让智能驾驶更安全。
InterFuserDVS: Event-Enhanced Sensor Fusion for Safe RL-Based Decision Making
- 将事件相机数据以令牌形式融合进视觉模型，增强动态环境感知
- 在CARLA基准上实现77.2分驾驶得分和100%路线完成率
- 适合关注自动驾驶感知鲁棒性与极端光照场景的研究者
自动驾驶系统高度依赖可靠的传感器融合来感知复杂环境。传统基于RGB相机和LiDAR的方案在高动态范围或高速场景中常因运动模糊和延迟而失效。动态视觉传感器（DVS，即事件相机）通过微秒级时间分辨率和高动态范围，异步捕捉亮度变化，带来范式革新。本文提出对前沿InterFuser模型的扩展架构，引入DVS作为额外模态，增强感知可靠性。我们设计了一种新的基于令牌的融合策略，将累积事件帧融入InterFuser的Transformer骨干网络。该方法充分利用了RGB、LiDAR与DVS数据的互补性。我们在Car Learning to Act（CARLA）Leaderboard基准上评估，结果表明引入DVS显著提升了驾驶智能体的鲁棒性，在复杂光照与动态条件下表现优异，取得77.2的驾驶得分和100%的路线完成率，验证了事件视觉在提升安全性与性能方面的巨大潜力。
原文摘要 · Abstract (English)
Autonomous driving systems rely heavily on robust sensor fusion to perceive complex envi- ronments. Traditional setups using RGB cameras and LiDAR often struggle in high-dynamic- range scenes or high-speed scenarios due to motion blur and latency. Dynamic Vision Sensors (DVS), or event cameras, offer a paradigm shift by capturing asynchronous brightness changes with microsecond temporal resolution and high dynamic range. In this paper, we propose an extended architecture of the state-of-the-art InterFuser model, integrating DVS as an additional modality to enhance perception reliability. We introduce a novel token-based fusion strategy that incorporates accumulated event frames into the transformer-based backbone of InterFuser. Our method leverages the complementary nature of RGB, LiDAR, and DVS data. We evaluate our approach on the Car Learning to Act (CARLA) Leaderboard benchmarks, demonstrating that the inclusion of DVS improves the robustness of the driving agent, achieving a competitive Driving Score of 77.2 and a superior Route Completion of 100%. The results indicate that event-based vision is a promising direction for improving safety and performance in adverse lighting and dynamic conditions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。