跨语言医学诊断新框架,零样本下对低资源语言表现优异
SwasthLLM: a Unified Cross-Lingual, Multi-Task, and Meta-Learning Zero-Shot Framework for Medical Diagnosis Using Contrastive Representations
- 用对比学习对齐多语言医疗文本语义,实现跨语言统一表示
- 零样本测试中对印地语达92.78%准确率,孟加拉语73.33%
- 无需微调即可快速适配新语言或任务,适合多语言医疗场景
在多语言医疗环境中,由于低资源语言标注数据稀缺及语言差异,临床文本自动疾病诊断仍具挑战。本文提出SwasthLLM,一个统一的零样本、跨语言、多任务元学习框架,可在英语、印地语和孟加拉语间有效运行,无需语言特定微调。核心采用增强语言感知注意力机制的多语言XLM-RoBERTa编码器,结合疾病分类头,提取跨语言医学信息。引入孪生对比学习模块,对齐不同语言间的语义表示;翻译一致性模块与对比投影头进一步强化语言不变表示学习。通过联合优化疾病分类、翻译对齐与对比学习目标的多任务训练策略,并使用模型无关元学习(MAML)赋予模型极小数据下的快速适应能力。分阶段训练优先强化表示对齐。评估显示,监督设置下测试准确率达97.22%,F1得分为97.17%;零样本下,印地语达92.78%准确率,孟加拉语73.33%,展现出强泛化能力。
原文摘要 · Abstract (English)
In multilingual healthcare environments, automatic disease diagnosis from clinical text remains a challenging task due to the scarcity of annotated medical data in low-resource languages and the linguistic variability across populations. This paper proposes SwasthLLM, a unified, zero-shot, cross-lingual, and multi-task learning framework for medical diagnosis that operates effectively across English, Hindi, and Bengali without requiring language-specific fine-tuning. At its core, SwasthLLM leverages the multilingual XLM-RoBERTa encoder augmented with a language-aware attention mechanism and a disease classification head, enabling the model to extract medically relevant information regardless of the language structure. To align semantic representations across languages, a Siamese contrastive learning module is introduced, ensuring that equivalent medical texts in different languages produce similar embeddings. Further, a translation consistency module and a contrastive projection head reinforce language-invariant representation learning. SwasthLLM is trained using a multi-task learning strategy, jointly optimizing disease classification, translation alignment, and contrastive learning objectives. Additionally, we employ Model-Agnostic Meta-Learning (MAML) to equip the model with rapid adaptation capabilities for unseen languages or tasks with minimal data. Our phased training pipeline emphasizes robust representation alignment before task-specific fine-tuning. Extensive evaluation shows that SwasthLLM achieves high diagnostic performance, with a test accuracy of 97.22% and an F1-score of 97.17% in supervised settings. Crucially, in zero-shot scenarios, it attains 92.78% accuracy on Hindi and 73.33% accuracy on Bengali medical text, demonstrating strong generalization in low-resource contexts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。