首个开放血液转录组数据多发性硬化分类基准,支持可复现研究。
MS-MLB: An Open Machine Learning Benchmark for Blood-Based MS Classification

- 构建基于GSE17048数据集的开源评估框架,控制数据泄露。
- 梯度提升模型在独立测试集上达AUC-ROC 0.989,F1值0.927。
- 适合机器学习研究者对比算法性能,不用于临床诊断。
多发性硬化(MS)诊断依赖临床评估、磁共振成像及实验室证据,并排除其他解释。血液中RNA表达数据可能包含与疾病相关的免疫信号,但血液RNA分类器不能替代临床诊断。本文提出MS-MLB(多发性硬化机器学习基准),一个基于全血RNA表达数据进行MS vs 健康对照分类的可复现开源基准。该基准使用公开的GSE17048队列,将其转化为MS vs 健康对照任务,采用统一且防泄露的评估流程,研究人员无需重新配置即可重跑。评估包括嵌套交叉验证、未触碰的分层保留集、自助置信区间、ROC与精确率-召回率分析、校准测量及探索性MS研究评分。最终基准结果中,梯度提升模型在保留集上以MS研究评分为93.83排名第一,对应AUC-ROC为0.989,敏感性0.950,特异性0.778,F1分数0.927,布里尔得分0.050。已有研究将机器学习应用于MS血液转录组数据,包括PBMC阶段分类和全血诊断特征建模。本工作的贡献在于首次针对GSE17048全血数据集构建专注的开源基准,并内置外部模型提交路径。该评分仅用于研究比较,未经临床验证。基准代码见:https://github.com/duckyquang/MS-MLB。
原文摘要 · Abstract (English)
Multiple sclerosis (MS) is diagnosed through clinical assessment, magnetic resonance imaging, laboratory evidence when appropriate, and exclusion of better explanations. Blood RNA expression data may contain disease associated immune signal, but a blood RNA classifier cannot be treated as a replacement for clinical diagnosis. This paper presents MS-MLB (Multiple Sclerosis Machine Learning Benchmark), a reproducible open benchmark for machine learning based MS research classification from whole blood RNA expression data. MS-MLB uses the public GSE17048 cohort, converts it into an MS versus healthy control task, and evaluates multiple algorithms under a shared, leakage controlled pipeline that a researcher can rerun without reconfiguring the evaluation. The evaluation includes nested cross-validation, an untouched stratified holdout set, bootstrap confidence intervals, ROC and precision recall analysis, calibration measurement, and an exploratory MS Research Score. In the final benchmark summary, Gradient Boosting ranked first by MS Research Score on the holdout set, with an MS Research Score of 93.83, AUC-ROC of 0.989, sensitivity of 0.950, specificity of 0.778, $F_{1}$ score of 0.927, and Brier score of 0.050. Prior studies have applied machine learning to MS blood transcriptomic data, including PBMC stage classification and whole blood diagnostic signature modeling. The contribution here is different and narrower. To our knowledge, MS-MLB is the first open benchmark focused on MS versus healthy control classification from GSE17048 whole blood RNA expression data with a documented external model submission pathway built into the framework. The score is intended for research comparison only and has not been clinically validated. The benchmark is accessible here: https://github.com/duckyquang/MS-MLB.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。