融合医生描述与乳腺影像，提升癌症检测准确率
Deep BI-RADS Network for Improved Cancer Detection from Mammograms
- 用迭代注意力融合影像与文本描述
- 在CBIS-DDSM数据集上各项指标显著提升
- 适合医学影像与多模态学习研究者
现有乳腺癌检测模型多仅依赖多视角乳腺影像，但放射科医生记录的病变文本描述包含额外有用信息。本研究提出一种新型多模态方法，将文本化的BI-RADS病变描述与视觉影像内容结合。通过迭代注意力层实现跨模态有效融合，在CBIS-DDSM数据集上的实验表明，相比仅使用图像的模型，该方法在所有评估指标上均有显著提升，验证了人工特征对端到端模型的贡献。
原文摘要 · Abstract (English)
While state-of-the-art models for breast cancer detection leverage multi-view mammograms for enhanced diagnostic accuracy, they often focus solely on visual mammography data. However, radiologists document valuable lesion descriptors that contain additional information that can enhance mammography-based breast cancer screening. A key question is whether deep learning models can benefit from these expert-derived features. To address this question, we introduce a novel multi-modal approach that combines textual BI-RADS lesion descriptors with visual mammogram content. Our method employs iterative attention layers to effectively fuse these different modalities, significantly improving classification performance over image-only models. Experiments on the CBIS-DDSM dataset demonstrate substantial improvements across all metrics, demonstrating the contribution of handcrafted features to end-to-end.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。