通过肿瘤位置加权的图文对比学习,提升儿童脑瘤诊断模型的可解释性。
Tumor Location-weighted MRI-Report Contrastive Learning: A Framework for Improving the Explainability of Pediatric Brain Tumor Diagnosis
- 融合MRI影像与放射报告,用对比学习提取可解释特征
- 注意力图与人工分割重合度达Dice 31.1%,分类准确率87.7%
- 适合关注临床可解释性的医学AI研究者与放射科医生
尽管卷积神经网络在磁共振成像(MRI)脑瘤诊断中表现优异,但其在临床流程中的应用受限,主要因模型预测所依赖的特征对放射科医生不透明,缺乏可解释性。放射科报告是专家知识的重要来源,可与MRI结合,在对比学习(CL)框架下实现图像-报告关联学习,从而提升CNN的可解释性。本文构建了一个多模态对比学习架构,基于3D脑部MRI和放射报告学习信息丰富的影像表征,并引入对多种脑瘤分析任务至关重要的肿瘤位置信息,以增强模型泛化能力。将学习到的图像表征应用于儿童低级别胶质瘤的基因标志物分类这一下游任务,结果显示模型注意力图与人工肿瘤分割的Dice分数达31.1%,分类测试性能达到87.7%,显著优于基线方法。该提升有助于建立放射科医生对模型的信任,推动其在临床中更高效地用于脑瘤诊断。
原文摘要 · Abstract (English)
Despite the promising performance of convolutional neural networks (CNNs) in brain tumor diagnosis from magnetic resonance imaging (MRI), their integration into the clinical workflow has been limited. That is mainly due to the fact that the features contributing to a model's prediction are unclear to radiologists and hence, clinically irrelevant, i.e., lack of explainability. As the invaluable sources of radiologists' knowledge and expertise, radiology reports can be integrated with MRI in a contrastive learning (CL) framework, enabling learning from image-report associations, to improve CNN explainability. In this work, we train a multimodal CL architecture on 3D brain MRI scans and radiology reports to learn informative MRI representations. Furthermore, we integrate tumor location, salient to several brain tumor analysis tasks, into this framework to improve its generalizability. We then apply the learnt image representations to improve explainability and performance of genetic marker classification of pediatric Low-grade Glioma, the most prevalent brain tumor in children, as a downstream task. Our results indicate a Dice score of 31.1% between the model's attention maps and manual tumor segmentation (as an explainability measure) with test classification performance of 87.7%, significantly outperforming the baselines. These enhancements can build trust in our model among radiologists, facilitating its integration into clinical practices for more efficient tumor diagnosis.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。