改进文本控制的潜在扩散模型,提升癌症病理图像生成质量与效率
Improving text-conditioned latent diffusion for cancer pathology
- 在潜在空间中结合变分自编码器与扩散模型,降低计算开销
- 生成病理图像FID达21.11,优于当前最优方法1.2分
- 训练时显存占用减少7%,适合大规模病理数据生成
过去十年生成模型的发展实现了超现实数据合成。尽管潜力巨大,但该技术在癌症组织病理学领域仍较少被探索。一种生成真实图像的方法是扩散模型,它通过迭代将图像转化为噪声并学习从噪声中恢复的过程。然而,对于高分辨率图像,该方法计算成本极高,难以应用于病理图像。变分自编码器(VAE)的发展使我们能够在潜在空间中学习复杂高分辨率图像的表示,并实现无损压缩与重建。将扩散模型与VAE结合,可在潜在空间中执行扩散,既保留了扩散模型的真实感生成能力,又保持合理的计算开销。Rombach等人和Yellapragada等人已为此任务建立基础模型。本文分析现有方法(如Yellapragada等,2023)的缺陷,修正关键错误并提出改进方案。实验表明,所提方法达到21.11的FID分数,优于前人工作1.2分,同时训练时GPU内存使用减少7%。
原文摘要 · Abstract (English)
The development of generative models in the past decade has allowed for hyperrealistic data synthesis. While potentially beneficial, this synthetic data generation process has been relatively underexplored in cancer histopathology. One algorithm for synthesising a realistic image is diffusion; it iteratively converts an image to noise and learns the recovery process from this noise [Wang and Vastola, 2023]. While effective, it is highly computationally expensive for high-resolution images, rendering it infeasible for histopathology. The development of Variational Autoencoders (VAEs) has allowed us to learn the representation of complex high-resolution images in a latent space. A vital by-product of this is the ability to compress high-resolution images to space and recover them lossless. The marriage of diffusion and VAEs allows us to carry out diffusion in the latent space of an autoencoder, enabling us to leverage the realistic generative capabilities of diffusion while maintaining reasonable computational requirements. Rombach et al. [2021b] and Yellapragada et al. [2023] build foundational models for this task, paving the way to generate realistic histopathology images. In this paper, we discuss the pitfalls of current methods, namely [Yellapragada et al., 2023] and resolve critical errors while proposing improvements along the way. Our methods achieve an FID score of 21.11, beating its SOTA counterparts in [Yellapragada et al., 2023] by 1.2 FID, while presenting a train-time GPU memory usage reduction of 7%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。