专注小病灶的多模态融合网络，提升肺病诊断准确率
Small Lesions-aware Bidirectional Multimodal Multiscale Fusion Network for Lung Disease Classification
- 构建双向多尺度注意力机制，精准捕捉3D影像小病灶特征
- 在Lung-PET-CT-Dx数据集上准确率超越现有方法
- 适合关注医学影像与电子病历融合的临床研究者
肺部疾病诊断面临小病灶误诊难题。深度学习，尤其是多模态方法，在医学诊断中展现出巨大潜力。然而，医学影像与电子健康记录数据在维度上的差异，给有效对齐与融合带来挑战。为此，我们提出多模态多尺度交叉注意力融合网络（MMCAF-Net）。该模型结合特征金字塔结构与高效的3D多尺度卷积注意力模块，从3D医学影像中提取病灶特异性特征。为进一步增强多模态数据融合，MMCAF-Net引入多尺度交叉注意力模块，解决维度不一致问题，实现更有效的特征融合。我们在Lung-PET-CT-Dx数据集上评估了该模型，结果表明诊断准确率显著提升，优于当前最先进方法。代码已开源：https://github.com/yjx1234/MMCAF-Net
原文摘要 · Abstract (English)
The diagnosis of medical diseases faces challenges such as the misdiagnosis of small lesions. Deep learning, particularly multimodal approaches, has shown great potential in the field of medical disease diagnosis. However, the differences in dimensionality between medical imaging and electronic health record data present challenges for effective alignment and fusion. To address these issues, we propose the Multimodal Multiscale Cross-Attention Fusion Network (MMCAF-Net). This model employs a feature pyramid structure combined with an efficient 3D multi-scale convolutional attention module to extract lesion-specific features from 3D medical images. To further enhance multimodal data integration, MMCAF-Net incorporates a multi-scale cross-attention module, which resolves dimensional inconsistencies, enabling more effective feature fusion. We evaluated MMCAF-Net on the Lung-PET-CT-Dx dataset, and the results showed a significant improvement in diagnostic accuracy, surpassing current state-of-the-art methods. The code is available at https://github.com/yjx1234/MMCAF-Net
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。