区分交叉验证与深度集成,避免误用不确定性估计。
Lost in the Folds: When Cross-Validation Is Not a Deep Ensemble for Uncertainty Estimation
- 用5折交叉验证构建的集成与深度集成方法对比实验
- 深度集成在校准和故障检测上表现更好,交叉验证更贴近医生间差异
- 适合可靠性评估时用深度集成,模糊性分析可用交叉验证
深度集成中的分歧常被用作医学图像分割中认知不确定性代理。实践中,许多研究通过5折交叉验证(CV)构建集成,却称之为“深度集成”(DE)。由于CV成员在不同数据子集上训练,其分歧混合了随机种子带来的变异与数据暴露效应,影响不确定性解释。我们审计了近期分割不确定性研究,发现术语与实现严重不匹配。随后在三个多评分者分割数据集(涵盖三种模态)上,将标准5折CV集成与5成员深度集成(固定训练集、不同随机种子)在相同配置下对比。评估包括校准性、故障检测、模糊性建模及分布偏移下的鲁棒性。结果显示:深度集成保持分割准确率的同时提升校准性与故障检测能力;而交叉验证集成在部分数据集上与评分者间变异性相关更强。因此,应根据研究目标选择集成方式:深度集成适用于可靠性导向任务(如选择性转诊/故障检测),交叉验证集成可作为模糊性的代理。我们提供轻量级nnU-Net修改,支持在默认流程中直接训练深度集成。
原文摘要 · Abstract (English)
Ensemble disagreement is widely used as a proxy for epistemic uncertainty in medical image segmentation. In practice, many studies form ensembles via K-fold cross-validation (CV), yet refer to them as ``deep ensembles'' (DE). Because CV members are trained on different data subsets, their disagreement mixes seed-driven variability with data-exposure effects, which can change how uncertainty should be interpreted. We audit recent segmentation uncertainty studies and find that terminology--implementation mismatches are common. We then compare a standard 5-fold CV ensemble to a 5-member DE (fixed training set, different random seeds) under otherwise identical configurations on three multi-rater segmentation datasets spanning three modalities. We evaluate uncertainty for calibration, failure detection, ambiguity modeling, and robustness under distribution shift. DE match segmentation accuracy while improving calibration and failure detection, whereas CV ensembles sometimes correlate more strongly with inter-rater variability on the studied datasets. Thus, ensemble construction should be chosen to match the research question: DE for reliability-oriented use (e.g., selective referral/failure detection) and CV ensembles as a proxy for ambiguity. We provide a lightweight nnU-Net modification enabling DE training within the default pipeline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。