用大模型提升四类疾病分类准确率，生物BERT表现最佳。
Enhancing Multi-Class Disease Classification: Neoplasms, Cardiovascular, Nervous System, and Digestive Disorders Using Advanced LLMs
- 对比BioBERT、XLNet等4个模型在医学文本上的分类表现。
- BioBERT达97%准确率，XLNet虽非医疗预训练也达96%。
- 轻量自研模型LastBERT性能接近标准BERT，适合资源受限场景。
本研究基于涵盖五种医学状况的Medical-Abstracts-TC-Corpus，探索预训练语言模型在多类别疾病分类中的改进效果。排除非癌症类疾病后，聚焦四种特定疾病进行分析。评估了BioBERT、XLNet、BERT及一种新提出的基线模型Last-BERT。结果显示，专为医学数据预训练的BioBERT在医学文本分类中表现最优，准确率达97%；令人意外的是，未在医学数据上预训练的XLNet也表现出色，准确率为96%，显示其跨领域泛化能力；而基于轻量版BERT构建的Last-BERT模型同样具备竞争力，准确率为87.10%，略低于BERT的89.33%。研究证实了专用模型如BioBERT的重要性，同时支持通用模型如XLNet和经过良好调优的轻量级Transformer架构在医学任务中的有效性。
原文摘要 · Abstract (English)
In this research, we explored the improvement in terms of multi-class disease classification via pre-trained language models over Medical-Abstracts-TC-Corpus that spans five medical conditions. We excluded non-cancer conditions and examined four specific diseases. We assessed four LLMs, BioBERT, XLNet, and BERT, as well as a novel base model (Last-BERT). BioBERT, which was pre-trained on medical data, demonstrated superior performance in medical text classification (97% accuracy). Surprisingly, XLNet followed closely (96% accuracy), demonstrating its generalizability across domains even though it was not pre-trained on medical data. LastBERT, a custom model based on the lighter version of BERT, also proved competitive with 87.10% accuracy (just under BERT's 89.33%). Our findings confirm the importance of specialized models such as BioBERT and also support impressions around more general solutions like XLNet and well-tuned transformer architectures with fewer parameters (in this case, LastBERT) in medical domain tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。