arXiv:2411.16792cs.CV2024-11被引 1

用2D扩散模型提升3D电镜图像分辨率,解决切片间不连贯问题。

From Diffusion to Resolution: Leveraging 2D Diffusion Models for 3D Super-Resolution Task

  • 用2D扩散模型逐层恢复横向切片,利用切片间连续性
  • 在两个FIB-SEM数据集上实现更优的分辨率与下游任务表现
  • 适合需要高精度3D结构重建的生物成像研究者

扩散模型在图像生成中表现出强大能力,尤其在图像超分辨率任务中。尽管2D扩散模型能显著提升单张图像分辨率,但现有基于扩散的3D体积超分辨率方法常面临轴向结构断裂和高采样成本的问题。本文提出一种新方法,利用2D扩散模型与体数据中的横向连续性,提升3D电子显微镜(vEM)体积超分辨率效果。首先在XY平面模拟横向退化,并训练2D扩散模型以恢复退化切片;随后在低分辨率体数据的横向方向上逐切片应用该模型,恢复切片并保持内在横向连续性;接着,使用高频感知的3D超分辨率网络对恢复后的横向切片序列进行训练,学习跨切片的空间特征变换;最后,将该网络应用于轴向推断,实现3D超分辨率。我们在两个公开的聚焦离子束扫描电镜(FIB-SEM)数据集上进行了全面评估,包括图像相似性、分辨率分析及下游任务表现。结果表明,该框架在鲁棒性和实际应用方面均具有优势。

原文摘要 · Abstract (English)

Diffusion models have recently emerged as a powerful technique in image generation, especially for image super-resolution tasks. While 2D diffusion models significantly enhance the resolution of individual images, existing diffusion-based methods for 3D volume super-resolution often struggle with structure discontinuities in axial direction and high sampling costs. In this work, we present a novel approach that leverages the 2D diffusion model and lateral continuity within the volume to enhance 3D volume electron microscopy (vEM) super-resolution. We first simulate lateral degradation with slices in the XY plane and train a 2D diffusion model to learn how to restore the degraded slices. The model is then applied slice-by-slice in the lateral direction of low-resolution volume, recovering slices while preserving inherent lateral continuity. Following this, a high-frequency-aware 3D super-resolution network is trained on the recovery lateral slice sequences to learn spatial feature transformation across slices. Finally, the network is applied to infer high-resolution volumes in the axial direction, enabling 3D super-resolution. We validate our approach through comprehensive evaluations, including image similarity assessments, resolution analysis, and performance on downstream tasks. Our results on two publicly available focused ion beam scanning electron microscopy (FIB-SEM) datasets demonstrate the robustness and practical applicability of our framework for 3D volume super-resolution.

3D超分辨率扩散模型电镜成像结构连续性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。