arXiv:2503.17222cs.CLcs.AI2025-03被引 2

用大模型自动判别心脑血管事件,提升临床试验效率与一致性。

Automating Adjudication of Cardiovascular Events Using Large Language Models

  • 分两阶段:先从病历中提取事件信息,再用思维树框架结合指南进行判断。
  • 事件抽取F1达0.82,判别准确率0.68,显著优于人工。
  • 提出新型评估指标CLEART,专用于衡量AI临床推理质量,适合医药研发者。

心脑血管事件(如心梗、中风)仍是全球主要死因,临床试验中需精细监测与判别。传统人工判别耗时长、成本高,且存在评审者间差异,可能引入偏倚并拖慢试验进度。本研究提出一种基于大语言模型(LLM)的自动化心血管事件判别框架。该框架分为两阶段:第一阶段利用LLM从非结构化临床数据中提取事件信息;第二阶段采用基于思维树(Tree of Thoughts)的LLM判别流程,并遵循临床终点委员会(CEC)指南。基于特定心血管临床试验数据,框架在事件提取上达到F1-score 0.82,判别准确率为0.68。此外,我们提出一种新指标CLEART,专门用于评估AI生成的临床推理质量。该方法显著降低判别时间和成本,保障高质量、一致且可审计的结果,同时加速对心血管治疗风险的识别与应对。

原文摘要 · Abstract (English)

Cardiovascular events, such as heart attacks and strokes, remain a leading cause of mortality globally, necessitating meticulous monitoring and adjudication in clinical trials. This process, traditionally performed manually by clinical experts, is time-consuming, resource-intensive, and prone to inter-reviewer variability, potentially introducing bias and hindering trial progress. This study addresses these critical limitations by presenting a novel framework for automating the adjudication of cardiovascular events in clinical trials using Large Language Models (LLMs). We developed a two-stage approach: first, employing an LLM-based pipeline for event information extraction from unstructured clinical data and second, using an LLM-based adjudication process guided by a Tree of Thoughts approach and clinical endpoint committee (CEC) guidelines. Using cardiovascular event-specific clinical trial data, the framework achieved an F1-score of 0.82 for event extraction and an accuracy of 0.68 for adjudication. Furthermore, we introduce the CLEART score, a novel, automated metric specifically designed for evaluating the quality of AI-generated clinical reasoning in adjudicating cardiovascular events. This approach demonstrates significant potential for substantially reducing adjudication time and costs while maintaining high-quality, consistent, and auditable outcomes in clinical trials. The reduced variability and enhanced standardization also allow for faster identification and mitigation of risks associated with cardiovascular therapies.

临床试验大模型医疗判别自动化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。