不同软件实现的C指数差异大,影响生存模型评估的可比性。
The C-index Multiverse
- 发现R和Python工具中相同C指数计算结果不一致
- 不同处理删失和并列情况导致结果偏差可达10%以上
- 适合做生存分析的科研人员和开发者参考
对生存时间预测模型的外部区分能力评估是预测建模中的关键步骤。常用指标为一致性指数(C-index),尤其在机器学习方法发展背景下更为普遍。我们发现,现有R和Python软件中存在‘C指数多世界’现象:看似相同的实现方式会产生不同结果,这会损害可复现性,并使模型与研究间的公平比较变得困难。主要差异来源包括对并列事件的处理方式及删失调整策略。此外,由于缺乏统一的风险汇总方法,输入类型不同也会引入额外变异。我们在公开的乳腺癌数据集及半合成案例上验证了多种生存模型(从Cox比例风险模型到深度学习方法)的性能评估受此影响。本文强调改进报告规范以提升透明度和可复现性,旨在为研究者提供统一文档与常见陷阱警示。所有代码已开源:www.github.com/BBolosSierra/CindexMultiverse。
原文摘要 · Abstract (English)
Quantifying out-of-sample discrimination performance for time-to-event outcomes is a fundamental step for model evaluation and selection in the context of predictive modelling. The concordance index, or C-index, is a widely used metric for this purpose, particularly with the growing development of machine learning methods. Beyond differences between proposed C-index estimators (e.g. Harrell's, Uno's and Antolini's), we demonstrate the existence of a C-index multiverse among available R and python software, where seemingly equal implementations can yield different results. This can undermine reproducibility and complicate fair comparisons across models and studies. Key variation sources include tie handling and adjustment to censoring. Additionally, the absence of a standardised approach to summarise risk from survival distributions, result in another source of variation dependent on input types. We demonstrate the consequences of the C-index multiverse when quantifying predictive performance for several survival models (from Cox proportional hazards to recent deep learning approaches) on publicly available breast cancer data, and semi-synthetic examples. Our work emphasises the need for better reporting to improve transparency and reproducibility. This article aims to be a useful guideline, helping analysts when navigating the multiverse, providing unified documentation and highlighting potential pitfalls of existing software. All code is publicly available at: www.github.com/BBolosSierra/CindexMultiverse.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。