融合影像与病理的AI系统提升肺癌诊断准确率并可解释结果。
Dual-Modal Lung Cancer AI: Interpretable Radiology and Microscopy with Clinical Risk Integration

- 用CT和病理图像双模态数据训练,结合临床信息做联合诊断。
- 准确率87%,AUROC超97%,对肿瘤区域定位精准。
- 支持医生理解AI判断依据,适合临床辅助决策场景。
肺癌仍是全球癌症致死的主要原因。传统CT影像虽用于检测与分期,但在良恶性鉴别及提供可解释诊断方面存在局限。本研究提出一种双模态人工智能框架,整合胸部CT影像与苏木精-伊红(H&E)组织病理图像,用于肺癌诊断与亚型分类。系统采用卷积神经网络提取影像特征,并引入临床元数据增强鲁棒性。双模态预测通过加权决策层融合机制进行整合,实现腺癌、鳞状细胞癌、大细胞癌、小细胞肺癌及正常组织的分类。应用Grad-CAM、Grad-CAM++、Integrated Gradients、Occlusion、Saliency Maps和SmoothGrad等可解释AI技术,提供可视化解释。实验显示系统表现优异:准确率最高达0.87,AUROC超过0.97,宏平均F1-score为0.88。其中Grad-CAM++在忠实度与定位精度上最优,与专家标注的肿瘤区域高度一致。结果表明,影像与病理的多模态融合可在保持模型透明性的前提下提升诊断性能,具备应用于精准肿瘤学临床决策支持系统的潜力。
原文摘要 · Abstract (English)
Lung cancer remains one of the leading causes of cancer-related mortality worldwide. Conventional computed tomography (CT) imaging, while essential for detection and staging, has limitations in distinguishing benign from malignant lesions and providing interpretable diagnostic insights. To address this challenge, this study proposes a dual-modal artificial intelligence framework that integrates CT radiology with hematoxylin and eosin (H&E) histopathology for lung cancer diagnosis and subtype classification. The system employs convolutional neural networks to extract radiologic and histopathologic features and incorporates clinical metadata to improve robustness. Predictions from both modalities are fused using a weighted decision-level integration mechanism to classify adenocarcinoma, squamous cell carcinoma, large cell carcinoma, small cell lung cancer, and normal tissue. Explainable AI techniques including Grad-CAM, Grad-CAM++, Integrated Gradients, Occlusion, Saliency Maps, and SmoothGrad are applied to provide visual interpretability. Experimental results show strong performance with accuracy up to 0.87, AUROC above 0.97, and macro F1-score of 0.88. Grad-CAM++ achieved the highest faithfulness and localization accuracy, demonstrating strong correspondence with expert-annotated tumor regions. These results indicate that multimodal fusion of radiology and histopathology can improve diagnostic performance while maintaining model transparency, suggesting potential for future clinical decision support systems in precision oncology.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。