让医生和AI轮流优化临床预测模型,提升准确性和实用性。
Human-AI Co-design for Clinical Prediction Models
- AI与医生交替迭代:AI快速挖掘病历中的医学概念,医生反馈优化。
- 在两种真实临床任务中表现优于现有方法,提升跨医院泛化能力。
- 适合需要可解释性、注重临床实用性的医疗AI研发团队。
构建安全、有效且实用的临床预测模型(CPMs)通常依赖临床专家、数据科学家和信息学家的反复协作,以精细调整模型构建中的关键细节,如特征/患者选择方式及临床分类定义。然而,这一传统协作过程极为耗时耗力,导致仅有少量CPMs能进入临床实践。当尝试整合非结构化病历文本时,该挑战进一步加剧,因其包含海量潜在概念。为此,我们提出HACHI——一种人机协同的迭代框架,利用AI代理加速可解释CPM的开发,支持对病历文本中概念的探索。HACHI在(i)AI代理快速筛选并评估病历中的候选概念,与(ii)临床和领域专家提供反馈以改进学习过程之间交替进行。概念被定义为简单的“是/否”问题,用于线性模型,使临床团队能够透明地审查、修正并验证每轮学习所得模型。在急性肾损伤和创伤性脑损伤两个真实预测任务中,HACHI优于现有方法,揭示了未被常用模型涵盖的新临床相关概念,并提升了模型在不同医疗机构和时间周期下的泛化能力。此外,该框架凸显了临床-人工智能团队的关键作用,包括引导AI探索新概念、调整概念粒度、改写目标函数以契合临床目标,以及识别数据偏差与泄漏问题。
原文摘要 · Abstract (English)
Developing safe, effective, and practically useful clinical prediction models (CPMs) traditionally requires iterative collaboration between clinical experts, data scientists, and informaticists. This process refines the often small but critical details of the model building process, such as which features/patients to include and how clinical categories should be defined. However, this traditional collaboration process is extremely time- and resource-intensive, resulting in only a small fraction of CPMs reaching clinical practice. This challenge intensifies when teams attempt to incorporate unstructured clinical notes, which can contain an enormous number of concepts. To address this challenge, we introduce HACHI, an iterative human-in-the-loop framework that uses AI agents to accelerate the development of fully interpretable CPMs by enabling the exploration of concepts in clinical notes. HACHI alternates between (i) an AI agent rapidly exploring and evaluating candidate concepts in clinical notes and (ii) clinical and domain experts providing feedback to improve the CPM learning process. HACHI defines concepts as simple yes-no questions that are used in linear models, allowing the clinical AI team to transparently review, refine, and validate the CPM learned in each round. In two real-world prediction tasks (acute kidney injury and traumatic brain injury), HACHI outperforms existing approaches, surfaces new clinically relevant concepts not included in commonly-used CPMs, and improves model generalizability across clinical sites and time periods. Furthermore, HACHI reveals the critical role of the clinical AI team, such as directing the AI agent to explore concepts that it had not previously considered, adjusting the granularity of concepts it considers, changing the objective function to better align with the clinical objectives, and identifying issues of data bias and leakage.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。