arXiv:2607.22984cs.SEcs.LG2026-07中稿 · the AIware 2026 ar…

用变异测试检测临床模型是否违背医学常识,发现高分模型仍可能出错。

Metamorphic Testing for Clinical ML Models: A Framework Proposal and Pilot Study

  • 基于临床指南设计12条变异关系,检验模型行为是否合理。
  • 3个模型的错误率高达27%至87%,即使AUROC超0.84。
  • 可发现传统指标忽略的特征错误,适合医疗AI验证场景。

用于临床预测(如院内死亡率、脓毒症发生)的机器学习模型常取得高AUROC分数,但该指标仅衡量排序能力,不反映临床合理性。模型可能整体排序正确,却在患者SOFA评分升高时预测死亡风险下降,违背医学常识。本文提出将变异测试(MT)应用于临床模型,评估其行为正确性,无需个体预测的真实标签。针对MIMIC-III和MIMIC-IV数据集中的三项重症监护任务,设计了12条基于权威临床指南的变异关系(MR),并提出五层验证策略确保其临床有效性。在UCI心脏病数据集上进行可行性研究:尽管三个模型的AUROC为0.849–0.900,但在五个试点MR中违反率介于27%至87%之间。注入故障实验显示,血压特征符号反转错误虽未影响AUROC,却使变异测试违规率提升31–67个百分点。结果表明,变异测试能有效补充传统指标,评估临床预测模型的行为正确性。

原文摘要 · Abstract (English)

Machine learning models for clinical prediction tasks, such as in-hospital mortality and sepsis onset, routinely achieve high AUROC scores. However, AUROC measures ranking performance rather than clinical sensibility. A model may rank patients correctly overall while predicting a lower mortality risk when a patient's SOFA score worsens, contradicting established medical knowledge. This paper proposes applying metamorphic testing (MT) to clinical machine learning models to evaluate behavioral correctness without requiring ground-truth labels for individual predictions. We design a catalog of 12 candidate metamorphic relations (MRs) for three ICU prediction tasks using the MIMIC-III and MIMIC-IV datasets, with each MR grounded in an authoritative clinical guideline. We further propose a five-layer validation strategy to ensure that MRs are clinically sound before deployment. As a feasibility study, we evaluate the approach on the UCI Heart Disease dataset. Although the three clinical models achieve strong predictive performance (AUROC = 0.849-0.900), they exhibit MT violation rates ranging from 27% to 87% across five pilot MRs. An injected-fault experiment further shows that a sign-negation error in a blood pressure feature remains undetected by AUROC but increases the MT violation rate by 31-67 percentage points. These findings suggest that metamorphic testing provides a valuable complement to conventional performance metrics for assessing the behavioral correctness of clinical prediction models.

医疗AI模型验证变异测试临床决策

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。