arXiv:2601.21955cs.CL2026-01被引 1

用轻量微调GPT-2分析病历文本,高效实现临床分类。

From Generative Modeling to Clinical Classification: A GPT-Based Architecture for EHR Notes

  • 只微调最后几层+分类头,大幅减少参数量。
  • 在MIMIC-IV数据集上多任务表现稳定,负向/未提及样本识别强。
  • 适合资源有限却需高精度病历分析的医疗团队。

电子健康记录(EHR)中非结构化临床文本日益丰富,为疾病表征、队列识别和临床决策支持提供了新机遇。然而,由于标注数据少、类别严重不均衡以及大规模预训练语言模型适配成本高,建模长篇领域特定临床文本仍具挑战。本文提出一种基于GPT的临床文本分类架构,采用选择性微调策略:冻结GPT-2主干大部分参数,仅训练最后的Transformer块、层归一化及轻量分类头。该方法显著降低可训练参数量,同时保留建模复杂临床语言所需的表征能力。在来自MIMIC-IV-Note数据集的放射科报告上评估,使用基于报告文本推导的不确定感知CheXpert式标签,涵盖多标签分类、单标签二分类(不同不确定性假设下)及整体疾病结局预测等多种任务。实验显示,该模型在不同数据规模下均表现出稳定收敛与优异分类性能,尤其在以未提及或否定发现为主的场景中表现突出。结果表明,对预训练生成式模型进行选择性微调,是实现高效、有效的临床文本分类的可行路径,可在显著降低计算复杂度的同时,实现对真实世界EHR数据的可扩展适配。

原文摘要 · Abstract (English)

The increasing availability of unstructured clinical narratives in electronic health records (EHRs) has created new opportunities for automated disease characterization, cohort identification, and clinical decision support. However, modeling long, domain-specific clinical text remains challenging due to limited labeled data, severe class imbalance, and the high computational cost of adapting large pretrained language models. This study presents a GPT-based architecture for clinical text classification that adapts a pretrained decoder-only Transformer using a selective fine-tuning strategy. Rather than updating all model parameters, the majority of the GPT-2 backbone is frozen, and training is restricted to the final Transformer block, the final layer normalization, and a lightweight classification head. This approach substantially reduces the number of trainable parameters while preserving the representational capacity required to model complex clinical language. The proposed method is evaluated on radiology reports from the MIMIC-IV-Note dataset using uncertainty-aware CheXpert-style labels derived directly from report text. Experiments cover multiple problem formulations, including multi-label classification of radiographic findings, binary per-label classification under different uncertainty assumptions, and aggregate disease outcome prediction. Across varying dataset sizes, the model exhibits stable convergence behavior and strong classification performance, particularly in settings dominated by non-mention and negated findings. Overall, the results indicate that selective fine-tuning of pretrained generative language models provides an efficient and effective pathway for clinical text classification, enabling scalable adaptation to real-world EHR data while significantly reducing computational complexity.

临床文本GPT-2轻量微调医疗AI

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。