用CLIP模型实现无需成对数据的医学图像跨域转换，效率更高。
I2I-Galip: Unsupervised Medical Image Translation Using Generative Adversarial CLIP
- 基于预训练CLIP模型，仅用一个轻量生成器完成多领域图像转换
- 在公开MRI和CT数据集上表现优于现有方法，参数量仅约1300万
- 适合需要高效跨模态医学图像转换的研究者或临床应用
无配对图像到图像的转换因源域与目标域分布差异大且缺乏成对样本而极具挑战性。现有主流方法如CycleGAN需为每对域单独训练生成器-判别器网络。本文提出I2I-Galip框架，利用预训练的多模态基础模型CLIP，避免为每对域单独训练生成器-判别器对，实现更高效、更优的多域图像转换。通过利用基础模型预训练积累的海量知识，本方法仅需一个参数量约1300万的轻量级生成器即可完成多域转换任务。在公开的MRI和CT数据集上的全面实验表明，该框架在转换性能上显著优于现有方法。代码将开源（https://github.com/yilmazkorkmaz1/I2I-Galip）。
原文摘要 · Abstract (English)
Unpaired image-to-image translation is a challenging task due to the absence of paired examples, which complicates learning the complex mappings between the distinct distributions of the source and target domains. One of the most commonly used approach for this task is CycleGAN which requires the training of a new pair of generator-discriminator networks for each domain pair. In this paper, we propose a new image-to-image translation framework named Image-to-Image-Generative-Adversarial-CLIP (I2I-Galip) where we utilize a pre-trained multi-model foundation model (i.e., CLIP) to mitigate the need of separate generator-discriminator pairs for each source-target mapping while achieving better and more efficient multi-domain translation. By utilizing the massive knowledge gathered during pre-training a foundation model, our approach makes use of a single lightweight generator network with ~13M parameters for the multi-domain image translation task. Comprehensive experiments on translation performance in public MRI and CT datasets show the superior performance of the proposed framework over the existing approaches. Code will be available (https://github.com/yilmazkorkmaz1/I2I-Galip).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。