用大模型自动提取临床试验中的累计医疗接触天数,提升效率与稳定性。
TimeTox: An LLM-Based Pipeline for Automated Extraction of Time Toxicity from Clinical Trial Protocols
- 分三阶段处理:摘要提取、时间毒性量化、多轮共识匹配
- 真实数据中95.3%结果在±3天内,82%完全稳定(四分位距为0)
- 适合需要高可复现性的临床研究自动化场景
时间毒性指临床试验参与带来的累计医疗接触天数,是重要但人工提取耗时的指标。我们开发了TimeTox,一个基于大模型的自动化提取管道,用于从评估时间表中提取时间毒性。该系统采用Google Gemini模型分三步进行:从完整协议文档中提取摘要、对每个治疗组在六个累积时间点量化时间毒性、通过位置匹配实现多轮一致性判断。在20个合成时间表(240次对比)上验证,两阶段架构达到100%临床可接受准确率(±3天),平均绝对误差0.81天;而单步架构仅41.5%准确,误差达9.0天。但在644份真实肿瘤学协议上,单步架构表现出更优可复现性:三次运行中95.3%结果在±3天内,四分位距≤3天,82.0%完全稳定(四分位距=0)。生产系统已为跨疾病领域的1,288个治疗组完成提取。真实数据上的稳定性,而非合成数据上的准确性,是大模型部署的关键考量。
原文摘要 · Abstract (English)
Time toxicity, the cumulative healthcare contact days from clinical trial participation, is an important but labor-intensive metric to extract from protocol documents. We developed TimeTox, an LLM-based pipeline for automated extraction of time toxicity from Schedule of Assessments tables. TimeTox uses Google's Gemini models in three stages: summary extraction from full-length protocol PDFs, time toxicity quantification at six cumulative timepoints for each treatment arm, and multi-run consensus via position-based arm matching. We validated against 20 synthetic schedules (240 comparisons) and assessed reproducibility on 644 real-world oncology protocols. Two architectures were compared: single-pass (vanilla) and two-stage (structure-then-count). The two-stage pipeline achieved 100% clinically acceptable accuracy ($\pm$3 days) on synthetic data (MAE 0.81 days) versus 41.5% for vanilla (MAE 9.0 days). However, on real-world protocols, the vanilla pipeline showed superior reproducibility: 95.3% clinically acceptable accuracy (IQR $\leq$ 3 days) across 3 runs on 644 protocols, with 82.0% perfect stability (IQR = 0). The production pipeline extracted time toxicity for 1,288 treatment arms across multiple disease sites. Extraction stability on real-world data, rather than accuracy on synthetic benchmarks, is the decisive factor for production LLM deployment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。