arXiv:2501.12488eess.IVcs.CV2025-01被引 6

用通用预训练模型迁移学习,实现脑部MRI与CT图像双向转换。

Bidirectional Brain Image Translation using Transfer Learning from Generic Pre-trained Models

  • 借用18个非医疗预训练模型,通过微调实现MRI与CT图像互转。
  • 在四项指标上表现优异,放射科医生评估也认可生成图像质量。
  • 适合缺乏标注数据的医学图像生成任务,尤其适用于脑成像研究。

脑成像在神经疾病诊断与治疗中至关重要,可提供脑结构与功能的宝贵信息。磁共振成像(MRI)和计算机断层扫描(CT)能无创可视化大脑,帮助理解解剖、异常及功能连接。但成本与辐射剂量限制了特定模态的获取,因此医学图像合成可生成所需图像而无需实际采集。在医学领域,标注图像获取耗时费力,数据稀缺是主要挑战。近期研究提出使用迁移学习解决此问题,即利用最初在非医疗数据上训练的预训练CycleGAN模型,生成真实医疗图像。本文将迁移学习应用于MR-CT图像互转任务,使用18个预训练非医疗模型,并进行微调以获得最佳效果。模型性能通过峰值信噪比(PSNR)、结构相似性指数(SSIM)、通用质量指数(UQI)和视觉信息保真度(VIF)四项常用图像质量指标评估。定量分析与放射科医生的定性感知评估均表明迁移学习在医学成像中的潜力及通用预训练模型的有效性。结果证明该模型表现卓越,归因于训练图像与真实人脑图像的高度相似性。这凸显了精心选择代表性训练图像对优化脑图像分析任务性能的重要性。

原文摘要 · Abstract (English)

Brain imaging plays a crucial role in the diagnosis and treatment of various neurological disorders, providing valuable insights into the structure and function of the brain. Techniques such as magnetic resonance imaging (MRI) and computed tomography (CT) enable non-invasive visualization of the brain, aiding in the understanding of brain anatomy, abnormalities, and functional connectivity. However, cost and radiation dose may limit the acquisition of specific image modalities, so medical image synthesis can be used to generate required medical images without actual addition. In the medical domain, where obtaining labeled medical images is labor-intensive and expensive, addressing data scarcity is a major challenge. Recent studies propose using transfer learning to overcome this issue. This involves adapting pre-trained CycleGAN models, initially trained on non-medical data, to generate realistic medical images. In this work, transfer learning was applied to the task of MR-CT image translation and vice versa using 18 pre-trained non-medical models, and the models were fine-tuned to have the best result. The models' performance was evaluated using four widely used image quality metrics: Peak-signal-to-noise-ratio, Structural Similarity Index, Universal Quality Index, and Visual Information Fidelity. Quantitative evaluation and qualitative perceptual analysis by radiologists demonstrate the potential of transfer learning in medical imaging and the effectiveness of the generic pre-trained model. The results provide compelling evidence of the model's exceptional performance, which can be attributed to the high quality and similarity of the training images to actual human brain images. These results underscore the significance of carefully selecting appropriate and representative training images to optimize performance in brain image analysis tasks.

图像生成迁移学习脑成像多模态

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。