arXiv:2501.01989cs.CVcs.AI2025-01被引 1

自动生成胸片报告并分类,提升放射科效率与准确性。

CRRG-CLIP: Automatic Generation of Chest Radiology Reports and Classification of Chest Radiographs

  • 用检测+分类+GPT生成报告,端到端完成诊断辅助。
  • 报告生成在多个指标上优于GPT-4o,分类准确率超现有模型。
  • 无需标注数据,利用未标注图文对训练,适合医疗场景部署。

复杂的影像堆叠和海量胸片导致报告撰写繁琐低效,即使经验丰富的放射科医生也难以在高强度工作下保持解读的准确性和一致性。为此,本文提出CRRG-CLIP模型(胸部放射学报告生成与胸片分类模型),一个端到端的自动化报告生成与分类模型。该模型包含两个模块:报告生成模块采用Faster R-CNN定位解剖区域,二分类器筛选关键区域,再用GPT-2生成语义连贯报告;分类模块则基于无监督对比语言图像预训练(CLIP)模型,克服标注数据成本高和特征不足的问题。实验表明,生成模块在BLEU、METEOR、ROUGE-L等指标上表现接近高性能基线模型,且在BLEU-2、BLEU-3、BLEU-4和ROUGE-L上优于GPT-4o;分类模块在AUC和准确率上显著超越当前最优模型。结果证明,该模型在报告生成中兼具高精度、可读性与流畅性,而利用未标注图文对进行多模态对比训练,有效提升了分类性能。

原文摘要 · Abstract (English)

The complexity of stacked imaging and the massive number of radiographs make writing radiology reports complex and inefficient. Even highly experienced radiologists struggle to maintain accuracy and consistency in interpreting radiographs under prolonged high-intensity work. To address these issues, this work proposes the CRRG-CLIP Model (Chest Radiology Report Generation and Radiograph Classification Model), an end-to-end model for automated report generation and radiograph classification. The model consists of two modules: the radiology report generation module and the radiograph classification module. The generation module uses Faster R-CNN to identify anatomical regions in radiographs, a binary classifier to select key regions, and GPT-2 to generate semantically coherent reports. The classification module uses the unsupervised Contrastive Language Image Pretraining (CLIP) model, addressing the challenges of high-cost labelled datasets and insufficient features. The results show that the generation module performs comparably to high-performance baseline models on BLEU, METEOR, and ROUGE-L metrics, and outperformed the GPT-4o model on BLEU-2, BLEU-3, BLEU-4, and ROUGE-L metrics. The classification module significantly surpasses the state-of-the-art model in AUC and Accuracy. This demonstrates that the proposed model achieves high accuracy, readability, and fluency in report generation, while multimodal contrastive training with unlabelled radiograph-report pairs enhances classification performance.

医学影像报告生成多模态CLIP

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。