用图文联合检测道路异常，减少误报。
Multi-modality Anomaly Segmentation on the Road
- 引入CLIP文本编码器实现图文融合，降低误判。
- 在三个数据集上优于现有方法，异常分割更准。
- 适合自动驾驶系统安全检测，尤其关注误报率。
语义分割使自动驾驶车辆能全面理解周围环境，但模型还需识别可能危及安全的障碍物。实验发现，当前单模态异常分割框架常对非异常区域产生过高异常分数。为此，我们提出多模态不确定性异常分割框架MMRAS+，首次为自动驾驶设计多模态异常分割方案。通过引入文本模态（使用CLIP文本编码器），有效降低非异常类别的高异常输出。此外，设计集成模块进一步提升性能。在RoadAnomaly、SMIYC和Fishyscapes验证数据集上的实验表明，该方法显著优于现有方案。代码已开源：https://github.com/HengGao12/MMRAS_plus。
原文摘要 · Abstract (English)
Semantic segmentation allows autonomous driving cars to understand the surroundings of the vehicle comprehensively. However, it is also crucial for the model to detect obstacles that may jeopardize the safety of autonomous driving systems. Based on our experiments, we find that current uni-modal anomaly segmentation frameworks tend to produce high anomaly scores for non-anomalous regions in images. Motivated by this empirical finding, we develop a multi-modal uncertainty-based anomaly segmentation framework, named MMRAS+, for autonomous driving systems. MMRAS+ effectively reduces the high anomaly outputs of non-anomalous classes by introducing text-modal using the CLIP text encoder. Indeed, MMRAS+ is the first multi-modal anomaly segmentation solution for autonomous driving. Moreover, we develop an ensemble module to further boost the anomaly segmentation performance. Experiments on RoadAnomaly, SMIYC, and Fishyscapes validation datasets demonstrate the superior performance of our method. The code is available in https://github.com/HengGao12/MMRAS_plus.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。