只微调GPT-2的末层,用6%参数实现91%医学文本分类准确率。
Selective Fine-Tuning of GPT Architectures for Parameter-Efficient Clinical Text Classification
- 仅更新模型末尾三部分,大幅减少可训练参数。
- 在5万份放射科报告上达到91%准确率,仅动用不足6%参数。
- 适合资源有限但需高效部署的临床文本分析场景。
电子病历系统生成了大量未结构化的临床文本,蕴含疾病识别、患者分组和临床决策支持的重要信息。但临床语言高度专业,标注数据稀缺,且全量微调大模型耗时耗力。本研究提出一种针对GPT-2的参数高效选择性微调框架,仅更新最后一层Transformer块、最后的层归一化模块和轻量分类头,其余参数冻结。该方法显著降低可训练参数量,同时保留预训练中的上下文表征能力。在包含5万份放射科报告的MIMIC-IV-Note数据集上,基于自动标注的CheXpert风格标签进行评估,仅更新少于6%参数即达约91%分类准确率。对比头部仅训练与全模型微调,本方法在性能与效率间取得更优平衡。结果表明,选择性微调为临床文本分类提供了高效且可扩展的解决方案。
原文摘要 · Abstract (English)
The rapid expansion of electronic health record (EHR) systems has generated large volumes of unstructured clinical narratives that contain valuable information for disease identification, patient cohort discovery, and clinical decision support. Extracting structured knowledge from these free-text documents remains challenging because clinical language is highly specialized, labeled datasets are limited, and full fine-tuning of large pretrained language models can require substantial computational resources. Efficient adaptation strategies are therefore essential for practical clinical natural language processing applications. This study proposes a parameter-efficient selective fine-tuning framework for adapting GPT-2 to clinical text classification tasks. Instead of updating the entire pretrained model, the majority of network parameters are frozen, and only the final Transformer block, the final layer normalization module, and a lightweight classification head are updated during training. This design substantially reduces the number of trainable parameters while preserving the contextual representation capabilities learned during pretraining. The proposed approach is evaluated using radiology reports from the MIMIC-IV-Note dataset with automatically derived CheXpert-style labels. Experiments on 50,000 radiology reports demonstrate that selective fine-tuning achieves approximately 91% classification accuracy while updating fewer than 6% of the model parameters. Comparative experiments with head-only training and full-model fine-tuning show that the proposed method provides a favorable balance between predictive performance and computational efficiency. These results indicate that selective fine-tuning offers an efficient and scalable framework for clinical text classification.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。