小模型实现多语言医疗数据匿名化,比大模型更准更安全
LLMs-in-the-Loop Part 2: Expert Small AI Models for Anonymization and De-identification of PHI Across Multiple Languages
- 用小模型+LLM协作框架,不传敏感数据也能做精准识别
- 八种语言平均F1达0.97以上,超越GPT-4o等通用大模型
- 适合医疗隐私保护、跨国研究等需高精度匿名的场景
慢性病与新冠疫情凸显了在保障隐私的前提下高效处理患者数据的迫切需求。通过匿名化和去标识化保护健康信息(PHI),可支持研究而不泄露患者身份。本文提出基于LLM-in-the-loop方法训练的专家级小型AI模型,用于多语言医疗文本的去标识化命名实体识别(NER)。这些模型避免了使用API调用大语言模型带来的隐私风险,无需传输或存储敏感数据。在英语、德语、意大利语、法语、罗马尼亚语、土耳其语、西班牙语和阿拉伯语共八种语言上,模型的f1-micro得分分别为0.966、0.975、0.976、0.970、0.964、0.974、0.978和0.953,均优于现有小型模型,甚至超过GPT-4o等通用大模型。该工作延续了第一篇关于生物医学文档翻译的研究,证明了该方法在低成本、高精度专业任务中的有效性,为未来医疗AI在实体与关系抽取等方向的发展奠定基础。
原文摘要 · Abstract (English)
The rise of chronic diseases and pandemics like COVID-19 has emphasized the need for effective patient data processing while ensuring privacy through anonymization and de-identification of protected health information (PHI). Anonymized data facilitates research without compromising patient confidentiality. This paper introduces expert small AI models developed using the LLM-in-the-loop methodology to meet the demand for domain-specific de-identification NER models. These models overcome the privacy risks associated with large language models (LLMs) used via APIs by eliminating the need to transmit or store sensitive data. More importantly, they consistently outperform LLMs in de-identification tasks, offering superior performance and reliability. Our de-identification NER models, developed in eight languages (English, German, Italian, French, Romanian, Turkish, Spanish, and Arabic) achieved f1-micro score averages of 0.966, 0.975, 0.976, 0.970, 0.964, 0.974, 0.978, and 0.953 respectively. These results establish them as the most accurate healthcare anonymization solutions, surpassing existing small models and even general-purpose LLMs such as GPT-4o. While Part-1 of this series introduced the LLM-in-the-loop methodology for bio-medical document translation, this second paper showcases its success in developing cost-effective expert small NER models in de-identification tasks. Our findings lay the groundwork for future healthcare AI innovations, including biomedical entity and relation extraction, demonstrating the value of specialized models for domain-specific challenges.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。