arXiv:2410.13896eess.IVcs.CV2024-10

提出抗伪影图像翻译框架,提升内镜视频与术前影像的对齐精度。

From Real Artifacts to Virtual Reference: A Robust Framework for Translating Endoscopic Images

  • 分步翻译:先局部去噪,再全局风格迁移
  • 对比学习提取抗噪特征,增强跨域对应关系
  • 在真实临床数据上显著优于现有方法

领域自适应在多模态医学图像分析中至关重要。在内镜成像中,结合术前数据与术中影像有助于手术规划与导航。然而,体内伪影导致的分布偏移限制了现有方法性能,亟需鲁棒技术将含噪声、伪影丰富的患者内镜视频与术前断层扫描重建的清晰虚拟图像对齐,以实现术中姿态估计。本文提出一种抗伪影图像翻译方法及相应基准。该方法采用新颖的“局部-全局”翻译框架:先局部步骤进行特征去噪,再全局步骤完成风格迁移;同时提出新型对比学习策略,提取抗噪特征以建立跨域稳健对应。在公开和自建临床数据集上的详尽验证表明,其性能显著优于当前最优方法。

原文摘要 · Abstract (English)

Domain adaptation, which bridges the distributions across different modalities, plays a crucial role in multimodal medical image analysis. In endoscopic imaging, combining pre-operative data with intra-operative imaging is important for surgical planning and navigation. However, existing domain adaptation methods are hampered by distribution shift caused by in vivo artifacts, necessitating robust techniques for aligning noisy and artifact abundant patient endoscopic videos with clean virtual images reconstructed from pre-operative tomographic data for pose estimation during intraoperative guidance. This paper presents an artifact-resilient image translation method and an associated benchmark for this purpose. The method incorporates a novel ``local-global'' translation framework and a noise-resilient feature extraction strategy. For the former, it decouples the image translation process into a local step for feature denoising, and a global step for global style transfer. For feature extraction, a new contrastive learning strategy is proposed, which can extract noise-resilient features for establishing robust correspondence across domains. Detailed validation on both public and in-house clinical datasets has been conducted, demonstrating significantly improved performance compared to the current state-of-the-art.

医学图像域自适应内镜图像翻译

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。