arXiv:2502.14189cs.CL2025-02被引 3

用四模型协作0样本分类医疗文本,准确率超80%

QUAD-LLM-MLTC: Large Language Models Ensemble Learning for Healthcare Text Multi-Label Classification

  • 集成GPT-4o、BERT等四模型,0样本下分步处理文本
  • 多标签分类F1达78.17%,微平均F1为80.16%
  • 适合需快速标注医疗文本的临床与研究场景

医疗文本数据量激增,但标注数据稀缺且语义复杂,传统机器学习难以全面捕捉主题。大语言模型(LLM)在自然语言处理中表现优异,尤其适用于无监督提示工程。为此,本文提出QUAD-LLM-MLTC方法,融合GPT-4o、BERT、PEGASUS和BART四模型:先由BERT提取关键词,再用PEGASUS增强文本,GPT-4o执行分类,BART输出标签概率,最终通过集成学习与元分类器生成结果。整个流程在0样本设置下运行。在三个标注数据集上评估显示,多数主题的F1分数显著提升,平均F1为78.17%(标准差0.025),微平均F1为80.16%(标准差0.011)。该方法为医疗文本多标签分类提供了高效可扩展的解决方案。

原文摘要 · Abstract (English)

The escalating volume of collected healthcare textual data presents a unique challenge for automated Multi-Label Text Classification (MLTC), which is primarily due to the scarcity of annotated texts for training and their nuanced nature. Traditional machine learning models often fail to fully capture the array of expressed topics. However, Large Language Models (LLMs) have demonstrated remarkable effectiveness across numerous Natural Language Processing (NLP) tasks in various domains, which show impressive computational efficiency and suitability for unsupervised learning through prompt engineering. Consequently, these LLMs promise an effective MLTC of medical narratives. However, when dealing with various labels, different prompts can be relevant depending on the topic. To address these challenges, the proposed approach, QUAD-LLM-MLTC, leverages the strengths of four LLMs: GPT-4o, BERT, PEGASUS, and BART. QUAD-LLM-MLTC operates in a sequential pipeline in which BERT extracts key tokens, PEGASUS augments textual data, GPT-4o classifies, and BART provides topics' assignment probabilities, which results in four classifications, all in a 0-shot setting. The outputs are then combined using ensemble learning and processed through a meta-classifier to produce the final MLTC result. The approach is evaluated using three samples of annotated texts, which contrast it with traditional and single-model methods. The results show significant improvements across the majority of the topics in the classification's F1 score and consistency (F1 and Micro-F1 scores of 78.17% and 80.16% with standard deviations of 0.025 and 0.011, respectively). This research advances MLTC using LLMs and provides an efficient and scalable solution to rapidly categorize healthcare-related text data without further training.

医疗文本多标签分类大模型集成0样本学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。