arXiv:2607.02091cs.CV2026-07

融合影像、临床与文本信息,提升乳腺纤维腺瘤与叶状肿瘤的精准分类

Multimodal Fusion for Fine-Grained Classification of Breast Fibroadenoma and Phyllodes Tumors

论文配图:Multimodal Fusion for Fine-Grained Classification of Breast Fibroadenoma and Phyllodes Tumors
图 1 · 摘自论文原文
  • 构建多模态数据集,融合超声图像、临床特征与诊断描述
  • 在910例患者上实现77.64%准确率,优于单一模态方法
  • 适合医学AI研究者及放射科医生参考应用

乳腺纤维腺瘤(FA)和叶状肿瘤(PT)在B型超声下表现高度相似,易导致良性及交界性PT被误判为FA,影响术前决策。现有辅助诊断方法多依赖单模态影像特征,未能充分挖掘临床与文本信息。为此,我们构建了FAPT-M数据集,包含910例经病理确诊的患者,涵盖严格审核的超声图像、结构化临床属性及超声诊断描述。基于此,提出一种临床引导的多模态框架,整合DenseNet视觉编码、CLIP式文本编码与轻量临床编码,并引入临床条件自适应调制、跨模态Transformer融合与双路径表征学习,增强特征对齐与多模态交互。在患者级五折交叉验证中,该方法达到77.64%准确率、73.38% F1-score与89.74% AUC,优于代表性CNN、Transformer及视觉语言基线模型。消融实验与类别平衡评估进一步验证三模态融合及关键组件的有效性。本工作为细粒度FA-PT分类提供有效方案,并建立高质量多模态乳腺超声分析基准。

原文摘要 · Abstract (English)

Breast fibroadenoma (FA) and phyllodes tumor (PT) are fibroepithelial breast lesions with highly overlapping appearances on B-mode ultrasound, making benign and borderline PT prone to being misclassified as FA and complicating preoperative decision-making. Existing computer-aided diagnosis methods commonly rely on single-modal imaging features and insufficiently exploit complementary clinical and textual information. To address this limitation, we construct the FAPT-M Dataset, a pathology-confirmed multimodal dataset comprising 910 patients with strictly reviewed ultrasound images, structured clinical attributes, and ultrasound diagnostic descriptions. Based on this dataset, we propose a clinically guided multimodal framework that integrates DenseNet-based visual encoding, CLIP-inspired text encoding, and lightweight clinical encoding, and further introduces clinical-conditioned adaptive modulation, cross-modal Transformer fusion, and dual-path representation learning to improve feature alignment and multimodal interaction. Under patient-level five-fold cross-validation, the proposed method achieves an accuracy of 77.64%, F1-score of 73.38%, and AUC of 89.74%, outperforming representative CNN-, Transformer-, and vision-language-based baselines. Ablation studies and class-balanced evaluations further confirm the contribution of three-modality fusion and the key architectural components. Overall, this work provides an effective multimodal approach for fine-grained FA-PT classification and establishes a high-quality benchmark for multimodal breast ultrasound analysis.

乳腺超声多模态融合细粒度分类

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。