用预训练模型特征实现跨模态医学图像配准,无需训练也能精准对齐。
IMPACT: A Generic Semantic Loss for Multimodal Medical Image Registration
- 基于分割模型提取的深层特征进行语义对比,不依赖原始像素或手工特征。
- 在5个三维配准任务中,目标配准误差和骰子系数均优于基线方法。
- 适用于临床与科研,兼容传统与深度学习框架,部署简单高效。
图像配准是医学影像中的基础任务,用于精确定位解剖结构,支持诊断、治疗规划、术中导航及长期监测。本文提出IMPACT(基于预训练模型无关比较的跨模态配准图像度量),一种新型相似性度量,专为鲁棒的多模态图像配准设计。与依赖原始强度、手工描述符或特定任务训练的方法不同,IMPACT通过比较来自大规模预训练分割模型(如TotalSegmentator、Segment Anything, SAM等)的深层特征,构建语义相似性度量。这些特征原本用于分割任务,具备强空间对应与语义对齐能力,天然适合配准。该方法可无缝集成至算法框架(Elastix)与学习型框架(VoxelMorph),融合两者优势。在包含胸腔CT/CBCT与骨盆MR/CT的5个挑战性3D配准任务中,定量评估显示目标配准误差与骰子相似系数均显著优于基线方法。定性分析进一步证明其在噪声、伪影与模态差异下的鲁棒性。凭借泛化性强、效率高、性能优,IMPACT为临床与科研场景中的多模态图像配准提供了强大解决方案。
原文摘要 · Abstract (English)
Image registration is fundamental in medical imaging, enabling precise alignment of anatomical structures for diagnosis, treatment planning, image-guided interventions, and longitudinal monitoring. This work introduces IMPACT (Image Metric with Pretrained model-Agnostic Comparison for Transmodality registration), a novel similarity metric designed for robust multimodal image registration. Rather than relying on raw intensities, handcrafted descriptors, or task-specific training, IMPACT defines a semantic similarity measure based on the comparison of deep features extracted from large-scale pretrained segmentation models. By leveraging representations from models such as TotalSegmentator, Segment Anything (SAM), and other foundation networks, IMPACT provides a task-agnostic, training-free solution that generalizes across imaging modalities. These features, originally trained for segmentation, offer strong spatial correspondence and semantic alignment capabilities, making them naturally suited for registration. The method integrates seamlessly into both algorithmic (Elastix) and learning-based (VoxelMorph) frameworks, leveraging the strengths of each. IMPACT was evaluated on five challenging 3D registration tasks involving thoracic CT/CBCT and pelvic MR/CT datasets. Quantitative metrics, including Target Registration Error and Dice Similarity Coefficient, demonstrated consistent improvements in anatomical alignment over baseline methods. Qualitative analyses further highlighted the robustness of the proposed metric in the presence of noise, artifacts, and modality variations. With its versatility, efficiency, and strong performance across diverse tasks, IMPACT offers a powerful solution for advancing multimodal image registration in both clinical and research settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。