通过不确定性感知训练提升自动驾驶分割在恶劣天气下的表现
Enhancing Self-Driving Segmentation in Adverse Weather Conditions: A Dual Uncertainty-Aware Training Approach to SAM Optimization
- 将不确定性度量引入SAM2的损失函数，实现多步微调优化
- UAT适配后在极端天气下分割精度显著优于标准SAM
- 适合关注自动驾驶安全与复杂环境鲁棒性的研究者
视觉基础模型如段落任意模型（SAM）及其后续版本SAM2在通用图像分割任务中表现优异，但在视觉模糊性高的恶劣天气条件下表现下降，主要因缺乏不确定性量化。受医学影像领域不确定性感知训练的启发，本文提出两种增强自动驾驶分割鲁棒性的方法：一是对SAM2采用多步微调，将不确定性度量直接嵌入损失函数，提升整体场景识别能力；二是将原用于医学图像分割的不确定性感知适配器（UAT）迁移至驾驶场景。在CamVid、BDD100K和GTA驾驶数据集上的实验表明，UAT-SAM在极端天气下表现优于标准SAM，而含不确定性感知损失的SAM2在多样化驾驶场景中均取得更优性能。结果证实显式不确定性建模对安全关键的自动驾驶在挑战性环境中的重要价值。
原文摘要 · Abstract (English)
Recent advances in vision foundation models, such as the Segment Anything Model (SAM) and its successor SAM2, have achieved state-of-the-art performance on general image segmentation benchmarks. However, these models struggle in adverse weather conditions where visual ambiguity is high, largely due to their lack of uncertainty quantification. Inspired by progress in medical imaging, where uncertainty-aware training has improved reliability in ambiguous cases, we investigate two approaches to enhance segmentation robustness for autonomous driving. First, we introduce a multi-step finetuning procedure for SAM2 that incorporates uncertainty metrics directly into the loss function, improving overall scene recognition. Second, we adapt the Uncertainty-Aware Adapter (UAT), originally designed for medical image segmentation, to driving contexts. We evaluate both methods on CamVid, BDD100K, and GTA driving datasets. Experiments show that UAT-SAM outperforms standard SAM in extreme weather, while SAM2 with uncertainty-aware loss achieves improved performance across diverse driving scenes. These findings underscore the value of explicit uncertainty modeling for safety-critical autonomous driving in challenging environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。