无需清晰参考图,模型可跨多种恶劣天气泛化,提升自动驾驶语义分割性能。
Generalizable Autonomous Driving System across Diverse Adverse Weather Conditions
- 基于短时天气不变性,设计无参考的抗恶劣天气框架
- 在多天气混合场景下实现88.56% mIoU提升
- 结合SAM与聚类算法,解决标注稀缺问题
多种恶劣天气对自动驾驶街景语义理解构成重大挑战。现有方法通常依赖清晰天气图像作为参考,但实际中难以获取,且多针对单一恶劣条件,在多重天气混合时表现不佳。为此,本文提出无参考、抗恶劣天气的方案AdvImmu,利用天气在短时间内的稳定性。AdvImmu包含三个组件:局部时序机制(LSM)利用相邻帧间的时序相关性增强性能;全局打乱机制(GSM)打乱LSM片段以防止时序模式过拟合;展开正则化器(URs)通过深度展开实现两种正则化,抑制模型复杂度以提升跨天气泛化能力。为缓解训练中对连续帧标注的依赖(实际中难获取),引入基础模型Segment Anything Model(SAM)辅助标注,并提出SBICAC聚类算法解决SAM类别无关问题,生成伪标签。大量实验表明,AdvImmu在均交并比(mIoU)上相较现有最优方法提升88.56%。
原文摘要 · Abstract (English)
Various adverse weather conditions pose a significant challenge to autonomous driving (AD) street scene semantic understanding (segmentation). A common strategy is to minimize the disparity between images captured in clear and adverse weather conditions. However, this technique typically relies on utilizing clear image as a reference, which is challenging to obtain in practice. Furthermore, this method typically targets a single adverse condition, and thus perform poorly when confronting a mixture of multiple adverse weather conditions. To address these issues, we introduce a reference-free and Adverse weather-Immune scheme (called AdvImmu) that leverages the invariance of weather conditions over short periods (seconds). Specifically, AdvImmu includes three components: Locally Sequential Mechanism (LSM), Globally Shuffled Mechanism (GSM), and Unfolded Regularizers (URs). LSM leverages temporal correlations between adjacent frames to enhance model performance. GSM is proposed to shuffle LSM segments to prevent overfitting of temporal patterns. URs are the deep unfolding implementation of two proposed regularizers to penalize the model complexity to enhance across-weather generalization. In addition, to overcome the over-reliance on consecutive frame-wise annotations in the training of AdvImmu (typically unavailable in AD scenarios), we incorporate a foundation model named Segment Anything Model (SAM) to assist to annotate frames, and additionally propose a cluster algorithm (denoted as SBICAC) to surmount SAM's category-agnostic issue to generate pseudo-labels. Extensive experiments demonstrate that the proposed AdvImmu outperforms existing state-of-the-art methods by 88.56% in mean Intersection over Union (mIoU).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。