用真实专家标注评估脊柱分割模型公平性,发现标签来源会误导结果。
False Confidence: Automated Labels Confound Fairness Audits in Cervical Spine Segmentation

- 用专家标注的金标准作为参考,避免机器生成标签干扰评估
- 同一模型在银标签下性能高估约8个Dice点,年龄公平性误判为显著差异
- 首次揭示标签来源是分割评估中的关键混淆因子,适合医疗AI公平性研究者
宫颈脊柱MRI自动分割在临床中日益普及,但尚未有针对该解剖结构的公平性审计。本研究发现,现代分割数据集普遍使用昂贵的专家标注(金标签)与大量低成本的机器生成标签(银标签)混合,而参考标签本身可能带有偏见。本文基于CSpineSeg数据集,首次对宫颈脊柱分割模型在性别、年龄和种族维度上的公平性进行审计。结果显示,部署模型在人口学上总体公平,但参考标签的选择并非中立:由于银标签由基于金标签训练的模型生成,新模型在金标签上表现接近专家真值,但在银标签上则更接近银标签本身。这种偏差导致性能被高估约8 Dice点,并将年龄公平性从非显著变为显著——这不是因差距膨胀(假幅度),而是因组内方差缩小(假信心)。因此,参考标签的来源是分割评估的一阶混淆因子:性能与公平性应以专家标签为基准,任何公平性结论都需注明参考标签来源。
原文摘要 · Abstract (English)
Automated segmentation of cervical-spine MRI is increasingly used in clinical workflows, yet no fairness audit exists for this anatomy. We show that auditing these segmentation tasks is complicated by a common property of modern segmentation datasets: expert-annotated gold labels are expensive, so abundant machine-generated (silver) labels are added to limit annotation cost. This matters because the reference used to judge a model can itself be biased. In this study, we present the first fairness audit of cervical-spine MRI segmentation across sex, age, and race using the CSpineSeg dataset. We observe that the deployed model is demographically fair, but the choice of reference label, however, is not neutral. Because a dataset's silver labels are generated by a model trained on its gold labels, any new model trained on those same gold labels agrees more with the silver labels than with expert truth: scoring identical predictions against silver rather than gold overestimates performance by ~8 Dice points and turns the fairness verdict for age from non-significant to significant - not by the gap inflation Parikh et al. report (which we term false magnitude) but by collapsing within-group variance (which we term false confidence). Reference-label provenance is thus a first-order confounder in segmentation evaluation: performance and fairness should be reported against expert labels, and any fairness claim stated together with the provenance of its reference.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。