用SAM模型提升医学图像配准的特征提取能力,效果显著优于现有方法。
SAMIR, an efficient registration framework via robust feature learning from SAM
- 基于SAM图像编码器构建适配任务的特征提取流水线,捕捉解剖结构信息。
- 在心脏和腹部数据集上分别提升2.68%和6.44%的配准准确率。
- 适合需要高精度配准且缺乏标注数据的医学影像研究者使用。
医学图像配准是医学影像分析的基础任务,其准确性依赖于对组织形态特征的精确建模。近期弱监督方法通过引入分割掩码或关键点等解剖先验来提升性能,但这些标签常难以获取,限制了实际应用。受视觉基础模型强大表征学习能力启发,本文提出SAMIR,一种基于分割一切模型(SAM)的高效医学图像配准框架。该方法不直接使用原始图像,而是利用SAM的预训练图像编码器提取结构感知的特征嵌入,并设计轻量级3D头以在嵌入空间中细化特征,适应局部形变。同时引入分层特征一致性损失,引导粗到精的特征匹配,提升解剖对齐效果。大量实验表明,SAMIR在跨被试腹部CT配准和同被试心脏图像配准任务中均显著超越现有方法,在ACDC数据集上提升2.68%,在腹部数据集上提升6.44%。代码将在论文录用后公开于GitHub。
原文摘要 · Abstract (English)
Image registration is a fundamental task in medical image analysis. Deformations are often closely related to the morphological characteristics of tissues, making accurate feature extraction crucial. Recent weakly supervised methods improve registration by incorporating anatomical priors such as segmentation masks or landmarks, either as inputs or in the loss function. However, such weak labels are often not readily available, limiting their practical use. Motivated by the strong representation learning ability of visual foundation models, this paper introduces SAMIR, an efficient medical image registration framework that utilizes the Segment Anything Model (SAM) to enhance feature extraction. SAM is pretrained on large-scale natural image datasets and can learn robust, general-purpose visual representations. Rather than using raw input images, we design a task-specific adaptation pipeline using SAM's image encoder to extract structure-aware feature embeddings, enabling more accurate modeling of anatomical consistency and deformation patterns. We further design a lightweight 3D head to refine features within the embedding space, adapting to local deformations in medical images. Additionally, we introduce a Hierarchical Feature Consistency Loss to guide coarse-to-fine feature matching and improve anatomical alignment. Extensive experiments demonstrate that SAMIR significantly outperforms state-of-the-art methods on benchmark datasets for both intra-subject cardiac image registration and inter-subject abdomen CT image registration, achieving performance improvements of 2.68% on ACDC and 6.44% on the abdomen dataset. The source code will be publicly available on GitHub following the acceptance of this paper.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。