提出X-RAFT模型，实现蓝光与白光神经外科高光谱图像的精准配准。
X-RAFT: Cross-Modal Non-Rigid Registration of Blue and White Light Neurosurgical Hyperspectral Images
- 采用双编码器架构，针对不同光照模态分别提取特征。
- 在神经外科数据上自监督微调，实现跨模态像素级对齐。
- 相比基线方法误差降低36.6%，适合实时荧光定量手术导航。
将高光谱成像引入荧光引导神经外科，可实现实时定量荧光测量，提升手术决策能力。定量荧光需同步获取荧光（蓝光）与反射（白光）模式下的配对光谱数据。由于蓝光与白光成像需顺序采集，在动态手术环境中易产生形变。关键挑战在于在极端光照差异下建立两幅高光谱图像间的密集跨模态对应关系。本文提出X-RAFT，基于递归全对场变换（RAFT）框架改进的跨模态光流模型，为每对模态设计独立图像编码器，并在神经外科高光谱数据上通过光流循环一致性进行自监督微调。实验显示，相较基线方法，评估指标平均误差降低36.6%；相比现有跨模态光流方法CrossRAFT，误差减少27.83%。代码与模型将在评审后公开。
原文摘要 · Abstract (English)
Integration of hyperspectral imaging into fluorescence-guided neurosurgery has the potential to improve surgical decision making by providing quantitative fluorescence measurements in real-time. Quantitative fluorescence requires paired spectral data in fluorescence (blue light) and reflectance (white light) mode. Blue and white image acquisition needs to be performed sequentially in a potentially dynamic surgical environment. A key component to the fluorescence quantification process is therefore the ability to find dense cross-modal image correspondences between two hyperspectral images taken under these drastically different lighting conditions. We address this challenge with the introduction of X-RAFT, a Recurrent All-Pairs Field Transforms (RAFT) optical flow model modified for cross-modal inputs. We propose using distinct image encoders for each modality pair, and fine-tune these in a self-supervised manner using flow-cycle-consistency on our neurosurgical hyperspectral data. We show an error reduction of 36.6% across our evaluation metrics when comparing to a naive baseline and 27.83% reduction compared to an existing cross-modal optical flow method (CrossRAFT). Our code and models will be made publicly available after the review process.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。