用多层级VQGAN从多重免疫荧光图像生成高保真虚拟H&E染色图
Cross-Domain Image Synthesis: Generating H&E from Multiplex Biomarker Imaging
- 采用多层级VQGAN生成虚拟H&E图像,保留分子空间信息
- 在结直肠癌数据集上,生成图像使后续核分割和组织分类准确率提升
- 适合需融合分子与形态分析的病理研究者使用
多重免疫荧光(mIF)成像可提供深度的空间分子信息,但将其与组织形态学标准——苏木精-伊红(H&E)染色结合,对理解组织背景至关重要。从mIF数据生成虚拟H&E染色可立即提供形态上下文,并使现有大量基于H&E的计算机辅助诊断(CAD)工具可用于分析丰富的分子数据,弥合分子与形态分析之间的鸿沟。本文研究了一种多层级向量量化生成对抗网络(VQGAN),用于从mIF图像生成高质量虚拟H&E染色。我们在两个公开的结直肠癌数据集上,将该VQGAN与标准条件生成对抗网络(cGAN)基线进行了严格对比,评估了图像相似性及下游分析的功能效用。结果表明,尽管两种架构均生成视觉合理的图像,但VQGAN生成的虚拟染色在后续核分割和组织分类任务中表现更优,与真实标注结果一致性更高。本工作证实,多层级VQGAN是生成科学可用虚拟染色的稳健且优越的架构,为将mIF的丰富分子数据整合到成熟的H&E分析流程中提供了可行路径。
原文摘要 · Abstract (English)
While multiplex immunofluorescence (mIF) imaging provides deep, spatially-resolved molecular data, integrating this information with the morphological standard of Hematoxylin & Eosin (H&E) can be very important for obtaining complementary information about the underlying tissue. Generating a virtual H&E stain from mIF data offers a powerful solution, providing immediate morphological context. Crucially, this approach enables the application of the vast ecosystem of H&E-based computer-aided diagnosis (CAD) tools to analyze rich molecular data, bridging the gap between molecular and morphological analysis. In this work, we investigate the use of a multi-level Vector-Quantized Generative Adversarial Network (VQGAN) to create high-fidelity virtual H&E stains from mIF images. We rigorously evaluated our VQGAN against a standard conditional GAN (cGAN) baseline on two publicly available colorectal cancer datasets, assessing performance on both image similarity and functional utility for downstream analysis. Our results show that while both architectures produce visually plausible images, the virtual stains generated by our VQGAN provide a more effective substrate for computer-aided diagnosis. Specifically, downstream nuclei segmentation and semantic preservation in tissue classification tasks performed on VQGAN-generated images demonstrate superior performance and agreement with ground-truth analysis compared to those from the cGAN. This work establishes that a multi-level VQGAN is a robust and superior architecture for generating scientifically useful virtual stains, offering a viable pathway to integrate the rich molecular data of mIF into established and powerful H&E-based analytical workflows.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。