提出新评估指标RSS,全面衡量分割模型的准确、可靠与鲁棒性。
Rethinking Semi-supervised Segmentation Beyond Accuracy: Reliability and Robustness
- 设计融合准确率、校准度与不确定性质量的综合评分RSS。
- 实验发现半监督模型常以牺牲可靠性换取精度提升。
- 适合关注实际部署安全性的自动驾驶等场景研究者。
语义分割对场景理解至关重要,但需昂贵的像素级标注,促使半监督方法利用大量未标注数据成为研究热点。然而当前评估仅关注分割精度,忽视了可靠性与鲁棒性。这两项特性对自动驾驶等安全关键应用至关重要,确保模型在多变环境下稳定运行并具备可信的置信度与不确定性估计。为此,本文提出可靠分割得分(RSS),通过调和平均整合预测准确率、校准度与不确定性质量。实验对比UniMatchV2与其前身及监督基线,发现半监督方法常以可靠性换精度;跨域测试显示UniMatchV2具鲁棒性,但仍存可靠性缺陷。建议采用如RSS等综合性指标重构评估体系,使半监督学习更贴近真实部署需求。
原文摘要 · Abstract (English)
Semantic segmentation is critical for scene understanding but demands costly pixel-wise annotations, attracting increasing attention to semi-supervised approaches to leverage abundant unlabeled data. While semi-supervised segmentation is often promoted as a path toward scalable, real-world deployment, it is astonishing that current evaluation protocols exclusively focus on segmentation accuracy, entirely overlooking reliability and robustness. These qualities, which ensure consistent performance under diverse conditions (robustness) and well-calibrated model confidences as well as meaningful uncertainties (reliability), are essential for safety-critical applications like autonomous driving, where models must handle unpredictable environments and avoid sudden failures at all costs. To address this gap, we introduce the Reliable Segmentation Score (RSS), a novel metric that combines predictive accuracy, calibration, and uncertainty quality measures via a harmonic mean. RSS penalizes deficiencies in any of its components, providing an easy and intuitive way of holistically judging segmentation models. Comprehensive evaluations of UniMatchV2 against its predecessor and a supervised baseline show that semi-supervised methods often trade reliability for accuracy. While out-of-domain evaluations demonstrate UniMatchV2's robustness, they further expose persistent reliability shortcomings. We advocate for a shift in evaluation protocols toward more holistic metrics like RSS to better align semi-supervised learning research with real-world deployment needs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。