发现3D医学影像知识迁移有强不对称性,提出按规律分配数据提升模型性能。
Knowledge Transfer Scaling Laws for 3D Medical Imaging

- 基于缩放定律优化跨模态数据分配,识别出可迁移性强的中心域与孤立域。
- 新策略相比等量采样提升58%性能,且在不同预算下泛化能力好(r=0.989)。
- 适合需要统一预训练的临床3D影像任务,如疾病分类与病灶分割。
视觉基础模型正从2D扩展至3D医学影像领域,统一预训练多种成像模态(如CT、MRI、PET)可为多样临床任务提供基础模型。然而,混合异构影像域训练时,当前数据混合策略仍多为启发式方法。本文发现不同医学影像域在预训练中具有不同的缩放速率,且域间知识迁移呈显著非对称性:一个域的训练可显著提升另一域,反之则可能效果甚微。有趣的是,MAE重建损失与跨域迁移均遵循可预测的幂律趋势,且具域特异性。受此启发,我们将数据分配建模为缩放律优化问题,推导出可解释的“枢纽-孤岛”结构:高可迁移域成为枢纽,惠及多个其他域,应优先投入;孤立域则需直接投资。实验表明,该迁移感知分配策略相比数据比例采样最高提升58%,并在未见预算下表现出优异泛化性(相关系数r=0.989)。下游任务验证显示,所获混合策略能为疾病分类与器官/病灶分割提供更强的预训练表征。
原文摘要 · Abstract (English)
Vision foundation models are increasingly moving beyond 2D to volumetric domains such as 3D medical imaging, where unified pretraining across different imaging modalities (i.e. CT, MRI, and PET) could provide foundational models for diverse clinical tasks. However, training such models requires mixing heterogeneous imaging domains, and current mixture strategies remain largely heuristic. In this work, we observe that different medical imaging domains scale at variable rates during pretraining, and knowledge transfer between domains is strongly asymmetric: training on one domain can substantially improve another, but the reverse may be much weaker. Interestingly, both MAE reconstruction loss and cross-domain transfer follow predictable power-law trends with domain-specific behaviors. Motivated by these findings, we formulate data allocation as a scaling-law optimization problem. The derived allocations reveal an interpretable hub-and-island structure: highly transferable domains emerge as hubs that benefit many others and deserve strategic allocation, while isolated domains act as islands requiring direct investment. Empirically, transfer-aware allocation outperforms data-proportional sampling by up to 58% and generalizes well to unseen budgets with r=0.989. Downstream validation on disease classification and organ/lesion segmentation further confirms that the derived transfer-aware mixtures provide stronger pretrained representations for clinical 3D medical imaging tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。