用视觉大模型提升医学影像配准精度,显式注入解剖结构信息。
Medical Image Registration Meets Vision Foundation Model: Prototype Learning and Contour Awareness
- 利用SAM生成分割掩码,显式注入解剖结构知识。
- 通过原型学习对齐语义对应关系,提升复杂结构配准效果。
- 设计轮廓感知损失,增强细粒度形变场建模能力。
医学图像配准是医学图像分析的基础任务,旨在建立配对图像间的空间对应关系。然而,现有无监督可变形配准方法仅依赖强度相似性度量,缺乏显式的解剖学知识,限制了其精度与鲁棒性。视觉基础模型(如分割任意模型SAM)可生成高质量分割掩码,提供明确的解剖结构信息,弥补传统方法仅依赖强度相似性的不足。为此,我们提出一种新型SAM辅助配准框架,融合原型学习与轮廓感知机制。该框架包括:(1) 显式解剖信息注入,将SAM生成的分割掩码作为训练与测试阶段的辅助输入,确保解剖一致性;(2) 原型学习,利用分割掩码提取原型特征,并对齐原型以优化图像间的语义对应关系;(3) 轮廓感知损失,基于分割掩码边缘设计损失函数,提升模型在细粒度形变场上的表现。大量实验表明,所提框架在多个数据集上显著优于现有方法,尤其在复杂解剖结构与模糊边界等挑战性场景中表现优异。代码已开源。
原文摘要 · Abstract (English)
Medical image registration is a fundamental task in medical image analysis, aiming to establish spatial correspondences between paired images. However, existing unsupervised deformable registration methods rely solely on intensity-based similarity metrics, lacking explicit anatomical knowledge, which limits their accuracy and robustness. Vision foundation models, such as the Segment Anything Model (SAM), can generate high-quality segmentation masks that provide explicit anatomical structure knowledge, addressing the limitations of traditional methods that depend only on intensity similarity. Based on this, we propose a novel SAM-assisted registration framework incorporating prototype learning and contour awareness. The framework includes: (1) Explicit anatomical information injection, where SAM-generated segmentation masks are used as auxiliary inputs throughout training and testing to ensure the consistency of anatomical information; (2) Prototype learning, which leverages segmentation masks to extract prototype features and aligns prototypes to optimize semantic correspondences between images; and (3) Contour-aware loss, a contour-aware loss is designed that leverages the edges of segmentation masks to improve the model's performance in fine-grained deformation fields. Extensive experiments demonstrate that the proposed framework significantly outperforms existing methods across multiple datasets, particularly in challenging scenarios with complex anatomical structures and ambiguous boundaries. Our code is available at https://github.com/HaoXu0507/IPMI25-SAM-Assisted-Registration.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。