arXiv:2605.07055cs.CVcs.AI2026-05被引 1

构建跨器官医学影像基础模型,解决数据缺失下的学习偏差问题。

Pan-FM: A Pan-Organ Foundation Model with Saliency-Guided Masking for Missing Robustness

论文配图:Pan-FM: A Pan-Organ Foundation Model with Saliency-Guided Masking for Missing Robustness
图 1 · 摘自论文原文
  • 用注意力引导的掩码机制动态屏蔽主导器官,平衡多器官学习
  • 在英国生物银行数据上,13类疾病预测性能超越单/多器官基线
  • 适用于系统神经科学等需要全身表征的医学影像研究

基础模型在医学影像中展现出巨大潜力,但多数模型仅基于单一器官的单模态数据训练。人类衰老与疾病涉及多器官协同过程,因此需要能学习全身表征的多模态基础模型。然而,真实世界多模态生物医学数据常存在非随机缺失,导致模型效能下降、泛化能力受限并引入偏差。本文提出Pan-FM,一个在七个器官(脑、心、脂肪、肝、肾、脾、胰腺)影像上预训练的跨器官基础模型,模拟真实世界的缺器官场景。该模型采用统一骨干网络,在训练和推理中均处理器官缺失,并通过基于掩码的自蒸馏方式预训练。我们发现,简单多模态预训练会引发主导器官捷径学习偏见,模型过度依赖脂肪和心脏等主导器官。为此,我们提出显著性引导掩码(SGM),利用模型注意力分布自适应地掩码主导器官,促进更均衡的跨器官、全身学习。值得注意的是,SGM计算开销极小,可无缝集成至现有自监督学习框架,提升多器官表征学习效果。在英国生物银行数据集上,Pan-FM在13类疾病及14个单一疾病实体的预测任务中表现优于单器官和多器官基线模型,且在缺器官设置下具备更强鲁棒性。Pan-FM为系统神经科学中的多模态学习提供可扩展的现实缺失解决方案,是迈向更具泛化性的全身基础模型的重要一步。

原文摘要 · Abstract (English)

Foundation models (FMs) have shown great promise in medical imaging, but most FMs are trained on unimodal data within isolated domains, such as brain MRI alone. Human aging and disease arise through coordinated biological processes across organs, therefore motivating multimodal FMs that learn whole-body representations. A key challenge, however, is that real-world multimodal biomedical data are often missing not at random, which can reduce power, limit generalizability, and introduce bias. We propose Pan-FM, a pan-organ foundation model pre-trained on imaging from seven organs (Brain, Heart, Adipose, Liver, Kidney, Spleen, and Pancreas) under realistic missing-organ scenarios. Pan-FM uses a unified backbone that handles organ missingness during both training and inference, and is pre-trained with masking-based self-distillation. We find that naive multimodal pre-training leads to dominant-organ shortcut learning bias, with the model over-relying on dominant organs such as adipose and heart. To address this, we introduce Saliency-Guided Masking (SGM), which uses the model attention distribution to adaptively mask dominant organs during pre-training, thus encouraging more balanced cross-organ, whole-body learning. Notably, SGM introduces negligible computational overhead and can be seamlessly integrated into existing self-supervised learning frameworks to improve multi-organ representation learning. On the UK Biobank, Pan-FM achieves stronger prediction across 13 disease categories and 14 single disease entities than single-organ and multi-organ baselines, with improved robustness under missing-organ settings. Pan-FM serves as a scalable solution to realistic modality-missingness in multimodal learning in system neuroscience and as a step toward more generalizable whole-body FMs.

基础模型多器官缺失数据医学影像

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。