arXiv:2501.07430cs.CVcs.AI2025-01被引 1

用2D扩散模型融合生成3D医学图像,提升精度与效率。

Introducing 3D Representation for Medical Image Volume-to-Volume Translation via Score Fusion

  • 通过在得分函数空间融合垂直训练的2D扩散模型,构建3D表示。
  • 在超分辨率和模态转换任务中,3D重建精度优于现有方法。
  • 适合需要高保真3D医学图像的科研与临床应用。

在医学图像体素到体素的转换中,现有模型因计算和数据需求高,难以有效捕捉三维体数据分布。本文提出Score-Fusion,一种新型体素转换模型,通过在得分函数空间集成垂直训练的2D扩散模型,高效学习3D表示。我们借鉴TPDM初始化方式,以2D模型平均值作为起点,将3D训练简化为微调过程,显著降低计算与数据需求。此外,模型显式设计分层结构以学习2D特征集合,进一步提升效率与性能。该方法自然扩展至多模态场景,通过融合不同输入条件下的扩散模型,实现灵活而精准的整合。实验表明,3D表示对下游识别任务(如肿瘤分割)至关重要,且大多数分割模型基于3D表示。大量实验证明,Score-Fusion在3D医学图像超分辨率与模态转换任务中,均达到更优的准确率与体素保真度。本工作还为基于得分函数融合的学习方法提供了新视角。

原文摘要 · Abstract (English)

In volume-to-volume translations in medical images, existing models often struggle to capture the inherent volumetric distribution using 3D voxelspace representations, due to high computational dataset demands. We present Score-Fusion, a novel volumetric translation model that effectively learns 3D representations by ensembling perpendicularly trained 2D diffusion models in score function space. By carefully initializing our model to start with an average of 2D models as in TPDM, we reduce 3D training to a fine-tuning process and thereby mitigate both computational and data demands. Furthermore, we explicitly design the 3D model's hierarchical layers to learn ensembles of 2D features, further enhancing efficiency and performance. Moreover, Score-Fusion naturally extends to multi-modality settings, by fusing diffusion models conditioned on different inputs for flexible, accurate integration. We demonstrate that 3D representation is essential for better performance in downstream recognition tasks, such as tumor segmentation, where most segmentation models are based on 3D representation. Extensive experiments demonstrate that Score-Fusion achieves superior accuracy and volumetric fidelity in 3D medical image super-resolution and modality translation. Beyond these improvements, our work also provides broader insight into learning-based approaches for score function fusion.

3D医学图像扩散模型图像翻译多模态融合

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。