用小模型+翻译提升低资源语言医疗信息提取效果
Small Language Models for Privacy-Preserving Clinical Information Extraction in Low-Resource Languages
- 先译后抽:用大模型把波斯语病历转成英文,再用小模型提取13类临床特征
- 70亿参数模型表现最好,平均F1达0.899,对生理症状提取尤其准
- 无需微调,适合资源有限的医院用,特别适合保护患者隐私
从低资源语言的医疗录音中提取临床信息仍是医疗NLP的重大挑战。本研究评估了一个两步流程:使用Aya-expanse-8B将1,221份匿名波斯语癌症姑息治疗热线录音翻译为英文,并结合五种开源小语言模型(Qwen2.5-7B-Instruct、Llama-3.1-8B-Instruct、Llama-3.2-3B-Instruct、Qwen2.5-1.5B-Instruct、Gemma-3-1B-it)进行13类临床特征的二分类抽取。采用少样本提示策略,未进行微调,通过宏平均F1、MCC、敏感性与特异性评估,以应对类别不平衡问题。结果表明,Qwen2.5-7B-Instruct综合表现最优(中位宏F1: 0.899;MCC: 0.797),而Gemma-3-1B-it最弱。70亿至80亿参数模型在敏感性和MCC上始终优于小模型。双语分析显示,将波斯语翻译为英文可提升敏感性、减少漏报,增强对类别不平衡的鲁棒性,但略降低特异性和精确率。特征层面结果显示,多数模型对生理症状提取可靠,心理诉求、行政请求及复杂体征仍具挑战。该研究为资源匮乏环境下部署开源小模型提供了一套隐私友好、可落地的方案,并强调模型规模与输入语言策略需协同优化。
原文摘要 · Abstract (English)
Extracting clinical information from medical transcripts in low-resource languages remains a significant challenge in healthcare natural language processing (NLP). This study evaluates a two-step pipeline combining Aya-expanse-8B as a Persian-to-English translation model with five open-source small language models (SLMs) -- Qwen2.5-7B-Instruct, Llama-3.1-8B-Instruct, Llama-3.2-3B-Instruct, Qwen2.5-1.5B-Instruct, and Gemma-3-1B-it -- for binary extraction of 13 clinical features from 1,221 anonymized Persian transcripts collected at a cancer palliative care call center. Using a few-shot prompting strategy without fine-tuning, models were assessed on macro-averaged F1-score, Matthews Correlation Coefficient (MCC), sensitivity, and specificity to account for class imbalance. Qwen2.5-7B-Instruct achieved the highest overall performance (median macro-F1: 0.899; MCC: 0.797), while Gemma-3-1B-it showed the weakest results. Larger models (7B--8B parameters) consistently outperformed smaller counterparts in sensitivity and MCC. A bilingual analysis of Aya-expanse-8B revealed that translating Persian transcripts to English improved sensitivity, reduced missing outputs, and boosted metrics robust to class imbalance, though at the cost of slightly lower specificity and precision. Feature-level results showed reliable extraction of physiological symptoms across most models, whereas psychological complaints, administrative requests, and complex somatic features remained challenging. These findings establish a practical, privacy-preserving blueprint for deploying open-source SLMs in multilingual clinical NLP settings with limited infrastructure and annotation resources, and highlight the importance of jointly optimizing model scale and input language strategy for sensitive healthcare applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。