arXiv:2603.01547cs.CV2026-03被引 1

融合病理图像、报告和细胞图谱,提升儿童脑瘤分类准确率并可解释。

PathMoE: Interpretable Multimodal Interaction Experts for Pediatric Brain Tumor Classification

  • 用专家混合架构动态融合三种医学模态信息,按输入自动分配权重。
  • 在内部数据集上宏平均F1达0.799,外部TCGA数据集提升至0.709。
  • 可解释性强,适合罕见肿瘤类型,利于临床医生信任与诊断验证。

儿童中枢神经系统肿瘤的精准分类因组织学复杂性和训练数据有限而困难。尽管病理基础模型已推进全切片图像(WSI)分析,但往往未能利用临床文本和组织微结构中的丰富互补信息。为此,我们提出PathMoE,一种可解释的多模态框架,通过基于各模态先进基础模型构建的交互感知专家混合架构,整合H&E染色切片、病理报告和核级细胞图谱。通过训练专用专家捕捉模态特异性、冗余与协同效应,PathMoE采用输入依赖的门控机制动态加权交互,实现样本级可解释性。我们在内部儿童脑瘤数据集(PBT)和外部TCGA数据集上评估该框架,分别进行两个任务。在PBT上,融合三模态使宏平均F1从0.762提升至0.799(+0.037);在TCGA上,引入图知识使宏平均F1从0.668提升至0.709(+0.041)。结果表明,相比最先进的仅图像基线,性能显著提升,同时揭示了驱动个体预测的具体模态交互。这种可解释性对罕见肿瘤亚型尤为重要,有助于建立临床信任与诊断验证。

原文摘要 · Abstract (English)

Accurate classification of pediatric central nervous system tumors remains challenging due to histological complexity and limited training data. While pathology foundation models have advanced whole-slide image (WSI) analysis, they often fail to leverage the rich, complementary information found in clinical text and tissue microarchitecture. To this end, we propose PathMoE, an interpretable multimodal framework that integrates H\&E slides, pathology reports, and nuclei-level cell graphs via an interaction-aware mixture-of-experts architecture built on state-of-the-art foundation models for each modality. By training specialized experts to capture modality uniqueness, redundancy, and synergy, PathMoE employs an input-dependent gating mechanism that dynamically weights these interactions, providing sample-level interpretability. We evaluate our framework on two dataset-specific classification tasks on an internal pediatric brain tumor dataset (PBT) and external TCGA datasets. PathMoE improves macro-F1 from 0.762 to 0.799 (+0.037) on PBT when integrating WSI, text, and graph modalities; on TCGA, augmenting WSI with graph knowledge improves macro-F1 from 0.668 to 0.709 (+0.041). These results demonstrate significant performance gains over state-of-the-art image-only baselines while revealing the specific modality interactions driving individual predictions. This interpretability is particularly critical for rare tumor subtypes, where transparent model reasoning is essential for clinical trust and diagnostic validation.

多模态病理分析可解释性儿童肿瘤

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。