基于3D CT的医学扩散模型,可通用处理多种医疗图像任务。
MedDiff-FM: A Diffusion-based Foundation Model for Versatile Medical Image Applications
- 用多部位3D CT数据预训练扩散模型,支持跨区域应用。
- 在去噪、异常检测、超分辨等任务上表现优异,支持快速微调。
- 适合需要多任务兼容的医学影像研究与临床开发人员。
扩散模型在自然图像和医学图像领域均取得显著进展,但以往研究多局限于特定解剖部位、特定任务和有限数据集,导致模型孤立。本文提出一种基于扩散的医学基础模型MedDiff-FM,利用来自多个公开数据集的3D CT图像(覆盖头至腹部),对扩散基础模型进行预训练,并探索其在多种应用场景下的能力。该模型在图像级与块级实现多层级集成处理,通过位置嵌入建立多层级空间关系,并利用区域类别和解剖结构捕捉特定解剖区域。MedDiff-FM可无缝处理多种下游任务,包括图像去噪、异常检测和图像合成。通过使用ControlNet结合任务特定条件,仅需快速微调即可实现超分辨率、病灶生成与病灶修复。实验结果表明,MedDiff-FM在多样化医学图像任务中具有显著有效性。
原文摘要 · Abstract (English)
Diffusion models have achieved significant success in both natural image and medical image domains, encompassing a wide range of applications. Previous investigations in medical images have often been constrained to specific anatomical regions, particular applications, and limited datasets, resulting in isolated diffusion models. This paper introduces a diffusion-based foundation model to address a diverse range of medical image tasks, namely MedDiff-FM. MedDiff-FM leverages 3D CT images from multiple publicly available datasets, covering anatomical regions from head to abdomen, to pre-train a diffusion foundation model, and explores the capabilities of the diffusion foundation model across a variety of application scenarios. The diffusion foundation model handles multi-level integrated image processing both at the image-level and patch-level, utilizes position embedding to establish multi-level spatial relationships, and leverages region classes and anatomical structures to capture certain anatomical regions. MedDiff-FM manages several downstream tasks seamlessly, including image denoising, anomaly detection, and image synthesis. MedDiff-FM is also capable of performing super-resolution, lesion generation, and lesion inpainting by rapidly fine-tuning the diffusion foundation model using ControlNet with task-specific conditions. The experimental results demonstrate the effectiveness of MedDiff-FM in addressing diverse downstream medical image tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。