arXiv:2411.03105cs.AIcs.LG2024-11

让机器学习模型更符合临床指南,提升可解释性与医疗连续性。

Evaluating Machine Learning Models against Clinical Protocols for Enhanced Interpretability and Continuity of Care

  • 设计新指标,评估模型预测与临床规则的一致性。
  • 融合临床规则的模型在准确性和一致性上优于纯数据模型。
  • 适合关注医疗AI落地与可解释性的研究者和临床开发者。

临床决策高度依赖既定协议(常以规则形式存在),而机器学习(ML)模型虽在临床数据上训练,却难以融入实际诊疗流程。核心挑战在于:(a)准确性——即使模型更精确,也可能引入协议不会产生的错误;(b)可解释性——黑箱模型可能基于违背临床知识的关系做判断。现有文献建议融合领域知识以提升准确率与可解释性,但缺乏有效评估方法。本文提出两种新指标:一是评估模型对临床协议的准确度;二是测量两类规则集解释之间的距离,以比较临床规则与模型提取规则的解释相似性。在皮马印第安人糖尿病数据集上,我们训练两个神经网络:一个仅依赖数据,另一个融合临床协议。结果表明,融合模型性能与纯数据模型相当,且对临床协议的准确率更高,保障了医疗连续性。同时,其预测解释更贴近临床规则,优于纯数据驱动模型。

原文摘要 · Abstract (English)

In clinical practice, decision-making relies heavily on established protocols, often formalised as rules. Concurrently, Machine Learning (ML) models, trained on clinical data, aspire to integrate into medical decision-making processes. However, despite the growing number of ML applications, their adoption into clinical practice remains limited. Two critical concerns arise, relevant to the notions of consistency and continuity of care: (a) accuracy - the ML model, albeit more accurate, might introduce errors that would not have occurred by applying the protocol; (b) interpretability - ML models operating as black boxes might make predictions based on relationships that contradict established clinical knowledge. In this context, the literature suggests using ML models integrating domain knowledge for improved accuracy and interpretability. However, there is a lack of appropriate metrics for comparing ML models with clinical rules in addressing these challenges. Accordingly, in this article, we first propose metrics to assess the accuracy of ML models with respect to the established protocol. Secondly, we propose an approach to measure the distance of explanations provided by two rule sets, with the goal of comparing the explanation similarity between clinical rule-based systems and rules extracted from ML models. The approach is validated on the Pima Indians Diabetes dataset by training two neural networks - one exclusively on data, and the other integrating a clinical protocol. Our findings demonstrate that the integrated ML model achieves comparable performance to that of a fully data-driven model while exhibiting superior accuracy relative to the clinical protocol, ensuring enhanced continuity of care. Furthermore, we show that our integrated model provides explanations for predictions that align more closely with the clinical protocol compared to the data-driven model.

机器学习临床决策可解释性医疗AI

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。