arXiv:2409.02045cs.CVcs.AI2024-09被引 2

统一增强恶劣天气与低光图像,提升自动驾驶视觉感知效果

AllWeatherNet:Unified Image Enhancement for Autonomous Driving under Adverse Weather and Lowlight-conditions

  • 分三层语义结构融合场景、物体与纹理信息,实现跨条件图像增强
  • 在训练域内使语义分割mIoU提升最高5.3%,未见域仍达3.9%提升
  • 光照感知注意力机制让模型对天气变化鲁棒,适合实际部署

雪、雨、夜间、雾等恶劣天气严重挑战自动驾驶感知系统。现有方法多仅针对单一场景(如去雨或夜转昼),且对语义分割等关键任务提升有限。本文提出AllWeather-Net,一种统一增强各类恶劣条件的图像增强方法。该方法采用分层架构,分别在场景、物体和纹理三个语义层级上区分并融合图像块信息;同时引入缩放光照感知注意力机制(SIAM),引导模型聚焦道路关键元素,具有强鲁棒性,不受天气或环境变化影响。AllWeather-Net能有效将图像转化为正常天气与白天场景,在训练域内使语义分割的mIoU最高提升5.3%;在未见域无需重训练即实现最高3.9%的mIoU增益。代码已开源。

原文摘要 · Abstract (English)

Adverse conditions like snow, rain, nighttime, and fog, pose challenges for autonomous driving perception systems. Existing methods have limited effectiveness in improving essential computer vision tasks, such as semantic segmentation, and often focus on only one specific condition, such as removing rain or translating nighttime images into daytime ones. To address these limitations, we propose a method to improve the visual quality and clarity degraded by such adverse conditions. Our method, AllWeather-Net, utilizes a novel hierarchical architecture to enhance images across all adverse conditions. This architecture incorporates information at three semantic levels: scene, object, and texture, by discriminating patches at each level. Furthermore, we introduce a Scaled Illumination-aware Attention Mechanism (SIAM) that guides the learning towards road elements critical for autonomous driving perception. SIAM exhibits robustness, remaining unaffected by changes in weather conditions or environmental scenes. AllWeather-Net effectively transforms images into normal weather and daytime scenes, demonstrating superior image enhancement results and subsequently enhancing the performance of semantic segmentation, with up to a 5.3% improvement in mIoU in the trained domain. We also show our model's generalization ability by applying it to unseen domains without re-training, achieving up to 3.9% mIoU improvement. Code can be accessed at: https://github.com/Jumponthemoon/AllWeatherNet.

图像增强自动驾驶语义分割跨域泛化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。