用检索增强的LLM从病历中自动找符合临床试验条件的患者。
Retrieval-Augmented LLMs for Evidence Localization in Clinical Trial Recruitment from Longitudinal EHR Narratives
- 用RAG技术动态检索证据,解决长病历中间信息丢失问题。
- MedGemma+RAG在基准测试中达到89.05%的微平均F1分数。
- 适合需要跨长文档推理的复杂入组标准筛选场景。
临床试验患者筛选是耗时费力的瓶颈,常导致入组不足和试验失败。本文系统研究了基于编码器和解码器的生成式大语言模型在临床病历中的应用,以提升筛选效率。比较了通用与医疗适配的LLM,并探索三种缓解长文档中“中间信息丢失”问题的策略:1)原始长上下文处理;2)基于命名实体识别的摘要提取;3)基于入选标准的动态证据检索(RAG)。使用2018年N2C2 Track 1数据集进行评估。结果表明,采用RAG策略的MedGemma模型取得最佳微平均F1分数89.05%,显著优于其他模型。生成式LLM对需长期推理的复杂标准提升明显,而短文本范围标准(如实验室检查)仅小幅改进。实际应用中应根据具体标准选择规则查询、编码器型或生成式模型,在计算成本可控下最大化效率。
原文摘要 · Abstract (English)
Screening patients for enrollment is a well-known, labor-intensive bottleneck that leads to under-enrollment and, ultimately, trial failures. Recent breakthroughs in large language models (LLMs) offer a promising opportunity to use artificial intelligence to improve screening. This study systematically explored both encoder- and decoder-based generative LLMs for screening clinical narratives to facilitate clinical trial recruitment. We examined both general-purpose LLMs and medical-adapted LLMs and explored three strategies to alleviate the "Lost in the Middle" issue when handling long documents, including 1) Original long-context: using the default context windows of LLMs, 2) NER-based extractive summarization: converting the long document into summarizations using named entity recognition, 3) RAG: dynamic evidence retrieval based on eligibility criteria. The 2018 N2C2 Track 1 benchmark dataset is used for evaluation. Our experimental results show that the MedGemma model with the RAG strategy achieved the best micro-F1 score of 89.05%, outperforming other models. Generative LLMs have remarkably improved trial criteria that require long-term reasoning across long documents, whereas trial criteria that span a short piece of context (e.g., lab tests) show incremental improvements. The real-world adoption of LLMs for trial recruitment must consider specific criteria for selecting among rule-based queries, encoder-based LLMs, and generative LLMs to maximize efficiency within reasonable computing costs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。