用深度学习消除雨天摄像头模糊，让自动驾驶看得更清。
Enhancing autonomous vehicle safety in rain: a data-centric approach for clear vision
- 基于编码器-解码器结构，区分雨滴高频纹理与场景低频信息。
- 在CARLA模拟中训练，雨天图像还原效果接近无雨清晰画面。
- 提升方向盘控制精度，适合车载视觉系统研发人员参考。
自动驾驶车辆在恶劣天气下面临严峻挑战，尤其雨天会严重干扰基于摄像头的视觉系统。本研究利用现代深度学习技术，开发一种视觉模型，可实时处理车载摄像头输入，去除雨滴造成的视觉干扰，生成接近无雨清晰场景的图像。研究基于Car Learning to Act（CARLA）仿真环境，构建了包含晴天与雨天图像的完整数据集用于模型训练与测试。模型采用经典的带跳跃连接和拼接操作的编码器-解码器架构，通过创新的分批策略，有效区分连续帧中的高频率雨纹与低频率场景特征。为评估性能，将模型集成至转向模块，以正前方图像为输入。结果表明，转向控制精度显著提升，验证了该模型在雨天提升自动驾驶导航安全性与可靠性方面的潜力。
原文摘要 · Abstract (English)
Autonomous vehicles face significant challenges in navigating adverse weather, particularly rain, due to the visual impairment of camera-based systems. In this study, we leveraged contemporary deep learning techniques to mitigate these challenges, aiming to develop a vision model that processes live vehicle camera feeds to eliminate rain-induced visual hindrances, yielding visuals closely resembling clear, rain-free scenes. Using the Car Learning to Act (CARLA) simulation environment, we generated a comprehensive dataset of clear and rainy images for model training and testing. In our model, we employed a classic encoder-decoder architecture with skip connections and concatenation operations. It was trained using novel batching schemes designed to effectively distinguish high-frequency rain patterns from low-frequency scene features across successive image frames. To evaluate the model performance, we integrated it with a steering module that processes front-view images as input. The results demonstrated notable improvements in steering accuracy, underscoring the model's potential to enhance navigation safety and reliability in rainy weather conditions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。