arXiv:2411.06106cs.CVcs.AI2024-11ICCV被引 5

通过学习个性化不变表示,提升多模态医学图像跨模态泛化能力

Towards a Universal 3D Medical Multi-modality Generalization via Learning Personalized Invariant Representation

  • 先学习个体级不变表征,再微调适应下游任务
  • 在多种场景下显著提升多模态任务泛化性能
  • 适合需要跨模态泛化的医学影像研究者

医学影像模态差异和个体解剖差异给多模态任务的跨模态泛化带来挑战。现有方法通常只关注共性解剖模式,忽略个体差异,限制了泛化性能。本文强调学习个体级不变表示(即个性化表征 $bX_h$)的重要性,以增强在同质与异质设置下的多模态泛化能力。研究表明,从个体生物特征到不同医学模态的映射在人群中保持稳定,这一特性体现在个性化过程中。我们提出两阶段方法:先通过不变表征 $bX_h$ 进行个性化预训练,再针对多样化下游任务进行微调。理论与实证结果均表明,个性化方法具有可行性与优势,在多种泛化场景下表现优于缺乏个性化的基线方法,显著提升跨多模态医学任务的通用性与可迁移性。

原文摘要 · Abstract (English)

Variations in medical imaging modalities and individual anatomical differences pose challenges to cross-modality generalization in multi-modal tasks. Existing methods often concentrate exclusively on common anatomical patterns, thereby neglecting individual differences and consequently limiting their generalization performance. This paper emphasizes the critical role of learning individual-level invariance, i.e., personalized representation $\mathbb{X}_h$, to enhance multi-modality generalization under both homogeneous and heterogeneous settings. It reveals that mappings from individual biological profile to different medical modalities remain static across the population, which is implied in the personalization process. We propose a two-stage approach: pre-training with invariant representation $\mathbb{X}_h$ for personalization, then fine-tuning for diverse downstream tasks. We provide both theoretical and empirical evidence demonstrating the feasibility and advantages of personalization, showing that our approach yields greater generalizability and transferability across diverse multi-modal medical tasks compared to methods lacking personalization. Extensive experiments further validate that our approach significantly enhances performance in various generalization scenarios.

医学影像多模态泛化能力个性化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。