用医学大模型嵌入提升跨机构医疗数据迁移效果
Transfer Learning with Clinical Concept Embeddings from Large Language Models
- 用领域专用大模型提取临床概念语义嵌入
- 医学专用模型在本地和迁移任务中表现更优
- 通用模型需微调，过度调参反而降低效果
知识共享在医疗领域至关重要，尤其在多机构联合应对数据稀缺、降低成本和实现及时干预时。迁移学习有助于跨机构知识传递，但主要挑战在于不同机构间临床概念的异质性。大型语言模型（LLMs）在捕捉临床概念语义并减少异质性方面展现出显著潜力。本研究分析了来自两个大型医疗系统的电子健康记录，评估了从大模型中提取的语义嵌入对本地模型、共享模型及迁移学习模型的影响。结果表明，领域特定的LLM如Med-BERT在本地和直接迁移场景中持续优于其他模型；而通用模型如OpenAI嵌入则需微调才能达到最佳性能。然而，对带有生物医学嵌入的模型进行过度微调会降低其有效性，凸显了平衡的重要性。该研究强调了领域专用嵌入与谨慎模型微调在医疗领域有效知识迁移中的关键作用。
原文摘要 · Abstract (English)
Knowledge sharing is crucial in healthcare, especially when leveraging data from multiple clinical sites to address data scarcity, reduce costs, and enable timely interventions. Transfer learning can facilitate cross-site knowledge transfer, but a major challenge is heterogeneity in clinical concepts across different sites. Large Language Models (LLMs) show significant potential of capturing the semantic meaning of clinical concepts and reducing heterogeneity. This study analyzed electronic health records from two large healthcare systems to assess the impact of semantic embeddings from LLMs on local, shared, and transfer learning models. Results indicate that domain-specific LLMs, such as Med-BERT, consistently outperform in local and direct transfer scenarios, while generic models like OpenAI embeddings require fine-tuning for optimal performance. However, excessive tuning of models with biomedical embeddings may reduce effectiveness, emphasizing the need for balance. This study highlights the importance of domain-specific embeddings and careful model tuning for effective knowledge transfer in healthcare.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。