arXiv:2604.06346cs.CLcs.AI2026-04被引 2

为阿拉伯语医疗文本生成设计了按病情严重度加权的损失函数,提升关键病例生成质量。

Severity-Aware Weighted Loss for Arabic Medical Text Generation

  • 基于病情严重度概率动态调整训练损失,优先优化高危病例。
  • 在MAQA数据集上,性能最高提升12.10%,跨模型效果稳定。
  • 无需改动模型结构,适用于各类阿拉伯语大模型,适合医疗场景应用。

大型语言模型在阿拉伯语医疗文本生成中展现出潜力,但传统微调目标对所有病例一视同仁,忽略了临床严重度差异。这一缺陷在医疗环境中尤为关键,因为严重病例中的错误风险更高。本文提出一种面向严重度的加权损失方法,用于在阿拉伯语医疗问诊数据上微调语言模型。该方法利用基于AraBERT的分类器自动获取病情严重度概率,并在优化过程中动态调整每个词元的损失权重,从而优先处理临床关键交互,且不修改模型架构。实验基于MAQA数据集进行,包含阿拉伯语医疗投诉与可信人工回复。在十种不同架构和参数规模的阿拉伯语大模型上评估,标准交叉熵微调仅带来有限提升,而严重度感知优化则持续获得显著增益:AraGPT2-Base从54.04%提升至66.14%,AraGPT2-Medium从59.16%升至67.18%,Qwen2.5-0.5B从57.83%增至66.86%,峰值达67.18%。整体来看,严重度感知微调相较非微调基线最高提升12.10%,展现稳健且一致的性能改进。

原文摘要 · Abstract (English)

Large language models have shown strong potential for Arabic medical text generation; however, traditional fine-tuning objectives treat all medical cases uniformly, ignoring differences in clinical severity. This limitation is particularly critical in healthcare settings, where errors in severe cases contain higher clinical risk. In this work, we propose a severity-aware weighted loss for fine-tuning Arabic language models on medical complaint-response data. The method depends on soft severity probabilities to dynamically scale token-level loss contributions during optimization, thereby prioritizing clinically critical interactions without modifying model architectures. Experiments are conducted using the MAQA dataset, which provides Arabic medical complaints and trusted human responses. Severity labels and probabilistic scores are automatically derived using a fine-tuned AraBERT-based classifier and incorporated exclusively at the loss level. The proposed approach is evaluated across ten Arabic large language models of varying architectures and parameter scales. While standard cross-entropy fine-tuning yields only modest improvements, severity-aware optimization consistently achieves larger gains. Using a balanced weighting configuration, performance improves from 54.04% to 66.14% for AraGPT2-Base, from 59.16% to 67.18% for AraGPT2-Medium, and from 57.83% to 66.86% for Qwen2.5-0.5B, with peak performance reaching 67.18%. Overall, severity-aware fine-tuning delivers improvements of up to 12.10% over non-fine-tuned baselines, demonstrating robust and architecture-consistent gains.

医疗生成加权损失阿拉伯语严重度感知

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。