用条件随机场提升3D医学图像生成的内存效率与质量
Memory-Efficient 3D High-Resolution Medical Image Synthesis Using CRF-Guided GANs
- 用条件随机场建模生成器中间特征的依赖关系
- 在低内存下实现高分辨率3D医学图像生成，无拼贴伪影
- 适用于医疗影像生成，尤其适合资源受限场景
生成对抗网络（GAN）在医学影像领域有广泛应用前景。由于GPU显存有限，当前大多数3D GAN模型仅在低分辨率医学图像上训练，难以扩展到高分辨率或易产生拼贴伪影。本文提出一种端到端新型GAN架构，利用条件随机场（CRF）建模依赖关系，实现在不增加内存负担的情况下生成一致的3D医学图像。训练时，生成器分为两部分：第一部分生成中间表示，随后应用CRF捕捉其相关性；第二部分基于中间表示的子集生成随机子体积图像。该结构具有双重优势：一是利用生成器优化过程中的特征建模相关性，二是推理时可生成完整高分辨率图像。在肺部CT和脑部MRI数据集上的实验表明，该方法在性能上超越现有先进方法，同时具备更低内存占用和更少计算复杂度。
原文摘要 · Abstract (English)
Generative Adversarial Networks (GANs) have many potential medical imaging applications. Due to the limited memory of Graphical Processing Units (GPUs), most current 3D GAN models are trained on low-resolution medical images, these models cannot scale to high-resolution or are susceptible to patchy artifacts. In this work, we propose an end-to-end novel GAN architecture that uses Conditional Random field (CRF) to model dependencies so that it can generate consistent 3D medical Images without exploiting memory. To achieve this purpose, the generator is divided into two parts during training, the first part produces an intermediate representation and CRF is applied to this intermediate representation to capture correlations. The second part of the generator produces a random sub-volume of image using a subset of the intermediate representation. This structure has two advantages: first, the correlations are modeled by using the features that the generator is trying to optimize. Second, the generator can generate full high-resolution images during inference. Experiments on Lung CTs and Brain MRIs show that our architecture outperforms state-of-the-art while it has lower memory usage and less complexity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。