用医学引导的多模态注意力模型,提升乳腺癌新辅助治疗后病理完全缓解预测精度。
Multimodal Stepwise Clinically-Guided Attention Learning for Pathological Complete Response Prediction in Breast Cancer

- 分步学习模拟医生推理:先学全局影像特征,再聚焦肿瘤区域,最后融合临床变量。
- 在外部数据集上敏感性显著提升,保持高特异性,且注意力图具解剖合理性。
- 适合关注可解释性与跨机构泛化的医学AI研究者和临床决策支持系统开发者。
病理完全缓解(pCR)是接受新辅助治疗的乳腺癌患者的重要预后指标,与长期生存率和个体化治疗密切相关。然而,由于类别严重不平衡以及在不同临床环境中的泛化能力有限,治疗前pCR预测仍具挑战。本文提出一种基于乳腺MRI的多模态分步临床引导注意力学习框架,通过医学启发的空间引导与多模态融合,解决上述问题。该方法遵循分步训练策略,模拟医生的推理过程:首先学习全局判别性影像模式,随后引入注意力机制聚焦肿瘤区域,最后整合临床变量优化决策。此策略促使模型优先关注任务相关特征,即使在响应者样本较少的情况下也能有效识别。同时,将注意力约束在解剖一致的肿瘤区域,减少对特定数据集模式的依赖,增强跨机构泛化能力。框架在异质性MRI队列中进行外部验证,相比非引导的单阶段基线,本方法在保持竞争性特异性的同时显著提升敏感性,并生成具有解剖合理性的注意力图,支持模型预测的可解释性。结果表明,临床引导的多模态注意力学习在乳腺癌pCR预测中具有鲁棒且可泛化的潜力。
原文摘要 · Abstract (English)
Pathological complete response (pCR) is a key prognostic factor in breast cancer patients undergoing neoadjuvant therapy, strongly associated with long-term survival and treatment personalization. However, accurate pre-treatment pCR prediction remains challenging due to severe class imbalance and limited generalizability across diverse clinical settings. In this work, we propose a multimodal stepwise clinically-guided attention learning framework for pCR prediction from breast magnetic resonance imaging (MRI), designed to address these limitations through medically grounded spatial guidance and multimodal integration. The approach follows a stepwise training strategy inspired by physician reasoning: the model first learns global discriminative imaging patterns, then attention mechanisms are introduced to constrain the network toward tumor regions, and finally clinical variables are integrated to refine decision-making. This guidance strategy encourages prioritization of task-relevant features, improving identification of responders despite their limited representation in the dataset. Moreover, grounding attention in anatomically consistent tumor regions reduces reliance on dataset-specific patterns, thereby enhancing cross-institutional generalization. The framework is evaluated through external validation across heterogeneous MRI cohorts. Compared to non-guided single-stage baselines, the proposed approach improves sensitivity while maintaining competitive specificity, and produces anatomically coherent attention maps that support interpretation of the model's predictions. These findings highlight the potential of clinically-guided multimodal attention learning for robust and generalizable pCR prediction in breast cancer.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。