arXiv:2507.20901cs.CV2025-07被引 1

用事件相机去除雪景图像中的雪花,提升自动驾驶视觉可靠性。

Event-Based De-Snowing for Autonomous Driving

  • 利用事件相机的时序特征捕捉雪花遮挡的轨迹痕迹,精准定位被遮区域。
  • 在新构建的DSEC-Snow数据集上,图像重建PSNR比现有方法高3dB。
  • 恢复后的图像可直接用于深度估计等任务,性能提升20%。

恶劣天气尤其是大雪对人类驾驶员和自动驾驶车辆构成重大挑战。传统基于图像的去雪方法仅依赖空间信息,易产生幻觉伪影;视频方法需高帧率,低帧率下会出现配准误差。相机曝光时间也影响雪花外观,使问题难以解决且依赖网络泛化能力。本文提出使用事件相机解决去雪问题,其具备毫秒级延迟与压缩视觉信息的优势,适合处理含自运动的场景。我们发现雪花遮挡在事件数据的时空表示中呈现独特条纹特征,设计注意力模块聚焦于这些条纹事件,判断背景点被遮挡时刻,并据此恢复原始亮度。我们在新构建的DSEC-Snow数据集上进行评估,该数据集通过绿屏技术将预录雪景叠加至DSEC驾驶数据集,生成精确标注的真值及同步图像与事件流。实验表明,本方法在图像重建上相较最先进方法提升3dB PSNR。此外,经本方法恢复的图像可直接用于深度估计与光流计算,性能较其他去雪方法提高20%。本工作为提升复杂冬季条件下视觉系统的可靠性与安全性迈出关键一步,推动全天候鲁棒应用发展。

原文摘要 · Abstract (English)

Adverse weather conditions, particularly heavy snowfall, pose significant challenges to both human drivers and autonomous vehicles. Traditional image-based de-snowing methods often introduce hallucination artifacts as they rely solely on spatial information, while video-based approaches require high frame rates and suffer from alignment artifacts at lower frame rates. Camera parameters, such as exposure time, also influence the appearance of snowflakes, making the problem difficult to solve and heavily dependent on network generalization. In this paper, we propose to address the challenge of desnowing by using event cameras, which offer compressed visual information with submillisecond latency, making them ideal for de-snowing images, even in the presence of ego-motion. Our method leverages the fact that snowflake occlusions appear with a very distinctive streak signature in the spatio-temporal representation of event data. We design an attention-based module that focuses on events along these streaks to determine when a background point was occluded and use this information to recover its original intensity. We benchmark our method on DSEC-Snow, a new dataset created using a green-screen technique that overlays pre-recorded snowfall data onto the existing DSEC driving dataset, resulting in precise ground truth and synchronized image and event streams. Our approach outperforms state-of-the-art de-snowing methods by 3 dB in PSNR for image reconstruction. Moreover, we show that off-the-shelf computer vision algorithms can be applied to our reconstructions for tasks such as depth estimation and optical flow, achieving a $20\%$ performance improvement over other de-snowing methods. Our work represents a crucial step towards enhancing the reliability and safety of vision systems in challenging winter conditions, paving the way for more robust, all-weather-capable applications.

事件相机去雪自动驾驶多模态感知

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。