用语义信息增强图像，提升复杂天气下目标检测精度。
SemOD: Semantic Enabled Object Detection Network under Various Weather Conditions
- 引入语义引导的图像修复模块，改善恶劣天气下的图像质量。
- 在多组天气数据集上，mAP提升1.47%至8.80%。
- 适合自动驾驶视觉感知、跨天气鲁棒检测的研究者。
在自动驾驶领域，基于摄像头的感知模型大多在晴天数据上训练。针对特定天气问题的方法难以适应多种天气变化，且主要关注去天气效果。本文提出一种语义增强的多天气目标检测网络（SemOD）。通过利用语义信息，模型可填补缺失区域、准确理解物体边界，并保持修复后与原图的视觉连贯性与真实感，有利于图像变换与目标识别。架构包含预处理单元（PPU）和检测单元（DTU），其中PPU采用带语义增强的U型网络修复退化图像，DTU则融合语义信息，使用改进的YOLO网络进行检测。该方法首次将语义数据用于全天气图像转换，在不同天气基准数据集上，相比现有方法，mAP提升1.47%至8.80%，验证了语义信息在图像增强与目标检测中的有效性，为提升检测性能提供了一种综合性解决方案。代码将公开于https://github.com/EnisZuo/SemOD。
原文摘要 · Abstract (English)
In the field of autonomous driving, camera-based perception models are mostly trained on clear weather data. Models that focus on addressing specific weather challenges are unable to adapt to various weather changes and primarily prioritize their weather removal characteristics. Our study introduces a semantic-enabled network for object detection in diverse weather conditions. In our analysis, semantics information can enable the model to generate plausible content for missing areas, understand object boundaries, and preserve visual coherency and realism across both filled-in and existing portions of the image, which are conducive to image transformation and object recognition. Specific in implementation, our architecture consists of a Preprocessing Unit (PPU) and a Detection Unit (DTU), where the PPU utilizes a U-shaped net enriched by semantics to refine degraded images, and the DTU integrates this semantic information for object detection using a modified YOLO network. Our method pioneers the use of semantic data for all-weather transformations, resulting in an increase between 1.47\% to 8.80\% in mAP compared to existing methods across benchmark datasets of different weather. This highlights the potency of semantics in image enhancement and object detection, offering a comprehensive approach to improving object detection performance. Code will be available at https://github.com/EnisZuo/SemOD.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。