用小模型在本地提取儿童病理报告信息,兼顾隐私与效率
A Semi-Automated Annotation Workflow for Paediatric Histopathology Reports Using Small Language Models
- 将临床实体提取转为问答任务,结合医生指导的规则和少量示例
- 小模型在仅用CPU的环境下达到84.3%准确率,优于多个主流模型
- 适合需要保护患者隐私、计算资源有限的医疗数据标注场景
电子病历系统包含宝贵临床信息,但多数以非结构化文本形式存在,限制了研究与决策应用。大语言模型虽可提取信息,但本地运行需大量算力,而将敏感数据上传云端即使脱敏仍引发隐私担忧。本研究开发了一种基于小语言模型(SLMs)的轻量级半自动化标注流程,用于从儿科病理报告中提取结构化信息。以肾活检报告为例,因其诊断范围有限且生物学基础明确,具备示范价值。研究通过三轮临床协作,人工标注2111份报告中的400份作为金标准,构建基于医生指导的实体规范与少样本示例的问答式抽取方法。采用分歧建模框架评估五种指令微调的SLMs,Gemma 2 2B表现最优,准确率达84.3%,显著优于spaCy(74.3%)、BioBERT-SQuAD(62.3%)、RoBERTa-SQuAD(59.7%)和GLiNER(60.2%)。实体指南使性能提升7-19%,少样本示例提升6-38%,但二者叠加无进一步增益。结果表明,小模型可在仅含CPU的设备上高效完成专业医学领域信息提取,且仅需极少临床参与。代码已开源:https://github.com/gosh-dre/nlp_renal_biopsy。
原文摘要 · Abstract (English)
Electronic Patient Record (EPR) systems contain valuable clinical information, but much of it is trapped in unstructured text, limiting its use for research and decision-making. Large language models can extract such information but require substantial computational resources to run locally, and sending sensitive clinical data to cloud-based services, even when deidentified, raises significant patient privacy concerns. In this study, we develop a resource-efficient semi-automated annotation workflow using small language models (SLMs) to extract structured information from unstructured EPR data, focusing on paediatric histopathology reports. As a proof-of-concept, we apply the workflow to paediatric renal biopsy reports, a domain chosen for its constrained diagnostic scope and well-defined underlying biology. We develop the workflow iteratively with clinical oversight across three meetings, manually annotating 400 reports from a dataset of 2,111 at Great Ormond Street Hospital as a gold standard, while developing an automated information extraction approach using SLMs. We frame extraction as a Question-Answering task grounded by clinician-guided entity guidelines and few-shot examples, evaluating five instruction-tuned SLMs with a disagreement modelling framework to prioritise reports for clinical review. Gemma 2 2B achieves the highest accuracy at 84.3%, outperforming off-the-shelf models including spaCy (74.3%), BioBERT-SQuAD (62.3%), RoBERTa-SQuAD (59.7%), and GLiNER (60.2%). Entity guidelines improved performance by 7-19% over the zero-shot baseline, and few-shot examples by 6-38%, though their benefits do not compound when combined. These results demonstrate that SLMs can extract structured information from specialised clinical domains on CPU-only infrastructure with minimal clinician involvement. Our code is available at https://github.com/gosh-dre/nlp_renal_biopsy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。