用单步去噪扩散GAN实现耳科手术场景补全,提升真实感。
SSDD-GAN: Single-Step Denoising Diffusion GAN for Cochlear Implant Surgical Scene Completion
- 融合扩散模型与GAN的单步去噪生成架构
- 在真实数据上自监督训练,零样本迁移至合成数据
- 相比基线提升结构相似性6%,适合手术导航研究
基于深度学习的图像修复方法在恢复受损图像方面表现出色,尤其在填补缺失区域方面。本文提出一种高效方法,用于补全合成的术后乳突切除术手术场景。该方法利用真实手术数据集进行自监督学习,训练出单步去噪扩散生成对抗网络(SSDD-GAN),结合了扩散模型与生成对抗网络的优势,在结构相似性上提升了6%。训练好的模型直接应用于合成的术后乳突切除术数据集,采用零样本方式生成真实且完整的手术场景,无需依赖该合成数据集的显式真值标签。该方法解决了先前工作无法还原周围手术环境的局限,为完整手术显微镜场景重建提供了新路径,增强了合成数据在术前规划和术中导航中的可用性。
原文摘要 · Abstract (English)
Recent deep learning-based image completion methods, including both inpainting and outpainting, have demonstrated promising results in restoring corrupted images by effectively filling various missing regions. Among these, Generative Adversarial Networks (GANs) and Denoising Diffusion Probabilistic Models (DDPMs) have been employed as key generative image completion approaches, excelling in the field of generating high-quality restorations with reduced artifacts and improved fine details. In previous work, we developed a method aimed at synthesizing views from novel microscope positions for mastoidectomy surgeries; however, that approach did not have the ability to restore the surrounding surgical scene environment. In this paper, we propose an efficient method to complete the surgical scene of the synthetic postmastoidectomy dataset. Our approach leverages self-supervised learning on real surgical datasets to train a Single-Step Denoising Diffusion-GAN (SSDD-GAN), combining the advantages of diffusion models with the adversarial optimization of GANs for improved Structural Similarity results of 6%. The trained model is then directly applied to the synthetic postmastoidectomy dataset using a zero-shot approach, enabling the generation of realistic and complete surgical scenes without the need for explicit ground-truth labels from the synthetic postmastoidectomy dataset. This method addresses key limitations in previous work, offering a novel pathway for full surgical microscopy scene completion and enhancing the usability of the synthetic postmastoidectomy dataset in surgical preoperative planning and intraoperative navigation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。