用大模型自动生成放射科报告，减少人工标注负担。
RadAnnotate: Large Language Models for Efficient and Reliable Radiology Report Annotation
- 基于检索增强生成合成报告，提升标注效率。
- 合成数据使不确定病灶识别F1提升至0.70，接近真实数据水平。
- 自动标注55%-90%报告，仅将低置信度样本交专家审核。
放射科报告标注对临床NLP至关重要，但人工标注耗时且成本高。本文提出RadAnnotate框架，通过检索增强的合成报告与基于置信度的自动化策略，降低RadGraph任务中专家标注负担。研究聚焦于实体标注（图节点），暂不涉及关系抽取（图边）。首先，在金标准报告上训练特定实体分类器，发现不确定描述最难学习。其次，生成RAG引导的合成报告，结果显示仅使用合成数据的模型在F1值上仅比真实数据训练模型低1-2点，且在低资源下对不确定观察的提升尤为显著，F1从0.61升至0.70。最后，通过学习各实体的置信度阈值，可自动标注55%-90%报告，保持0.86-0.92的实体匹配分数，同时将低置信度样本送交专家复核。
原文摘要 · Abstract (English)
Radiology report annotation is essential for clinical NLP, yet manual labeling is slow and costly. We present RadAnnotate, an LLM-based framework that studies retrieval-augmented synthetic reports and confidence-based selective automation to reduce expert effort for labeling in RadGraph. We study RadGraph-style entity labeling (graph nodes) and leave relation extraction (edges) to future work. First, we train entity-specific classifiers on gold-standard reports and characterize their strengths and failure modes across anatomy and observation categories, with uncertain observations hardest to learn. Second, we generate RAG-guided synthetic reports and show that synthetic-only models remain within 1-2 F1 points of gold-trained models, and that synthetic augmentation is especially helpful for uncertain observations in a low-resource setting, improving F1 from 0.61 to 0.70. Finally, by learning entity-specific confidence thresholds, RadAnnotate can automatically annotate 55-90% of reports at 0.86-0.92 entity match score while routing low-confidence cases for expert review.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。