在医疗影像报告生成中,小数据下微调大模型易过拟合,需谨慎选择模型大小。
Deep learning and abstractive summarisation for radiological reports: an empirical study for adapting the PEGASUS models' family with scarce data
- 用PEGASUS系列模型在小样本放射科报告上微调,对比不同规模检查点表现。
- 小模型在训练中出现双下降现象,大模型反而因参数过多导致性能下降。
- 为医学领域摘要生成提供避免过/欠拟合的实证指导,适合研究医疗AI的开发者。
尽管人工智能发展迅速,但敏感且数据受限的医学领域仍面临抽象摘要生成的挑战。随着影像数据增加,自动化复杂医学文本摘要工具的重要性日益凸显。本文研究了非领域专用的抽象摘要编码器-解码器模型家族(PEGASUS与PEGASUS-X)在小规模放射科报告公开数据集上的微调适应过程,为实践者提供避免过拟合与欠拟合的洞见。对每种模型,我们使用相同训练数据的不同规模检查点进行了全面评估,并在固定大小验证集上监控训练过程中的词汇与语义指标表现。结果显示,PEGASUS表现出不同阶段特征,可能对应于按轮次的双下降或峰值下降-恢复行为;而PEGASUS-X中更大的检查点反而导致性能下降。本工作揭示了在稀缺数据下使用高表达力模型微调时的挑战与风险,为未来更稳健的专用领域摘要模型微调策略奠定了基础。
原文摘要 · Abstract (English)
Regardless of the rapid development of artificial intelligence, abstractive summarisation is still challenging for sensitive and data-restrictive domains like medicine. With the increasing number of imaging, the relevance of automated tools for complex medical text summarisation is expected to become highly relevant. In this paper, we investigated the adaptation via fine-tuning process of a non-domain-specific abstractive summarisation encoder-decoder model family, and gave insights to practitioners on how to avoid over- and underfitting. We used PEGASUS and PEGASUS-X, on a medium-sized radiological reports public dataset. For each model, we comprehensively evaluated two different checkpoints with varying sizes of the same training data. We monitored the models' performances with lexical and semantic metrics during the training history on the fixed-size validation set. PEGASUS exhibited different phases, which can be related to epoch-wise double-descent, or peak-drop-recovery behaviour. For PEGASUS-X, we found that using a larger checkpoint led to a performance detriment. This work highlights the challenges and risks of fine-tuning models with high expressivity when dealing with scarce training data, and lays the groundwork for future investigations into more robust fine-tuning strategies for summarisation models in specialised domains.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。