用合成超声图像训练3D跨模态关键点描述符,实现磁共振与术中超声的精准配准。
A 3D Cross-modal Keypoint Descriptor for MR-US Matching and Registration
- 通过从MRI生成合成超声图,实现监督对比学习,构建共享描述子空间。
- 在11名患者上平均匹配精度达69.8%,注册误差均值2.39毫米,优于现有方法。
- 无需手动初始化,对超声视野变化和斑点噪声鲁棒,适合临床实时应用。
术中实时超声(iUS)与术前磁共振(MRI)的配准因成像模态差异大、分辨率和视野不一致而难以实现。为此,本文提出一种用于MRI-iUS匹配与配准的新型3D跨模态关键点描述符。方法采用患者特异性“以合成匹配”策略,从术前MRI生成合成iUS体积,实现监督对比学习,构建共享描述子空间。通过概率性关键点检测识别解剖显著且模态一致的位置。训练阶段使用基于课程学习的三元组损失与动态难负样本挖掘,使描述符具备对iUS斑点噪声、覆盖范围有限等伪影的鲁棒性及旋转不变性。推理时,在MR和真实iUS图像中检测关键点并识别稀疏匹配,用于刚性配准。在ReMIND数据集的3D MRI-iUS对上评估,本方法在11名患者上平均精度达69.8%;在ReMIND2Reg基准上注册误差均值为2.39毫米,性能优于当前最优方法。相比现有方法,本框架可解释性强,无需手动初始化,对iUS视野变化具有鲁棒性。代码、数据与模型权重已开源。
原文摘要 · Abstract (English)
Intraoperative registration of real-time ultrasound (iUS) to preoperative Magnetic Resonance Imaging (MRI) remains an unsolved problem due to severe modality-specific differences in appearance, resolution, and field-of-view. To address this, we propose a novel 3D cross-modal keypoint descriptor for MRI-iUS matching and registration. Our approach employs a patient-specific matching-by-synthesis approach, generating synthetic iUS volumes from preoperative MRI. This enables supervised contrastive training to learn a shared descriptor space. A probabilistic keypoint detection strategy is then employed to identify anatomically salient and modality-consistent locations. During training, a curriculum-based triplet loss with dynamic hard negative mining is used to learn descriptors that are i) robust to iUS artifacts such as speckle noise and limited coverage, and ii) rotation-invariant. At inference, the method detects keypoints in MR and real iUS images and identifies sparse matches, which are then used to perform rigid registration. Our approach is evaluated using 3D MRI-iUS pairs from the ReMIND dataset. Experiments show that our approach outperforms state-of-the-art keypoint matching methods across 11 patients, with an average precision of 69.8%. For image registration, our method achieves a competitive mean Target Registration Error of 2.39 mm on the ReMIND2Reg benchmark. Compared to existing iUS-MR registration approaches, our framework is interpretable, requires no manual initialization, and shows robustness to iUS field-of-view variation. Code, data and model weights are available at https://github.com/morozovdd/CrossKEY.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。