用小模型在本地生成医疗记录,既保护隐私又省成本。
Preserving Privacy, Increasing Accessibility, and Reducing Cost: An On-Device Artificial Intelligence Model for Medical Transcription and Note Generation
- 用低资源方法微调10亿参数模型,在浏览器内完成医疗转录
- 关键指标提升显著:错误率降53%,正确性评分升26%
- 适合关注数据安全、算力有限的医疗机构使用
临床文档撰写给医护人员带来沉重负担,医生每天需花费近2小时处理行政事务。尽管大语言模型有潜力解决此问题,但隐私顾虑和计算开销限制了其在医疗场景的应用。本文开发并评估了一种基于微调后的Llama 3.2 1B模型的本地化医疗转录系统,可在浏览器中完全离线运行,生成结构化病历。通过在1,500对合成医患对话-结构化病历数据上使用参数高效微调(LoRA)进行训练,并在100个内分泌科转录与140个修改版ACI基准测试集上评估。采用ROUGE、BERTScore、BLEURT等统计指标及大模型作为评判者,从多个临床质量维度进行测评。结果显示,微调后模型性能显著提升:在ACI基准上,ROUGE-1从0.346增至0.496,BERTScore F1从0.832升至0.866;重大幻觉案例从85例降至35例,事实正确性评分从2.81升至3.54(5分制)。内部数据集综合评分由3.13升至4.43,提升41.5%。结论表明,微调轻量级LLM可实现临床有效改进,同时支持全本地部署,解决了医疗AI应用中的隐私、成本与可访问性三大障碍。
原文摘要 · Abstract (English)
Background: Clinical documentation represents a significant burden for healthcare providers, with physicians spending up to 2 hours daily on administrative tasks. Recent advances in large language models (LLMs) offer promising solutions, but privacy concerns and computational requirements limit their adoption in healthcare settings. Objective: To develop and evaluate a privacy-preserving, on-device medical transcription system using a fine-tuned Llama 3.2 1B model capable of generating structured medical notes from medical transcriptions while maintaining complete data sovereignty entirely in the browser. Methods: We fine-tuned a Llama 3.2 1B model using Parameter-Efficient Fine-Tuning (PEFT) with LoRA on 1,500 synthetic medical transcription-to-structured note pairs. The model was evaluated against the base Llama 3.2 1B on two datasets: 100 endocrinology transcripts and 140 modified ACI benchmark cases. Evaluation employed both statistical metrics (ROUGE, BERTScore, BLEURT) and LLM-as-judge assessments across multiple clinical quality dimensions. Results: The fine-tuned OnDevice model demonstrated substantial improvements over the base model. On the ACI benchmark, ROUGE-1 scores increased from 0.346 to 0.496, while BERTScore F1 improved from 0.832 to 0.866. Clinical quality assessments showed marked reduction in major hallucinations (from 85 to 35 cases) and enhanced factual correctness (2.81 to 3.54 on 5-point scale). Similar improvements were observed on the internal evaluation dataset, with composite scores increasing from 3.13 to 4.43 (+41.5%). Conclusions: Fine-tuning compact LLMs for medical transcription yields clinically meaningful improvements while enabling complete on-device browser deployment. This approach addresses key barriers to AI adoption in healthcare: privacy preservation, cost reduction, and accessibility for resource-constrained environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。