解决多光谱医学影像跨波段对应难题,提升融合精度。
Cross-Spectral Dense Correspondence for Multimodal Spectral Medical Imaging

- 设计传感器无关的跨光谱调制协议,生成合成训练数据。
- 在严重光谱失配下性能显著提升,保持标准RGB任务表现。
- 适用于异构医疗光谱系统,推动高光谱成像融合应用。
精确的密集对应是融合不同波长范围的多模态光谱成像系统的基础,用于后续医学与科学成像分析。由于对应像素点常具有非重叠的光谱敏感性,导致波长相关的对比度变化、强度反转和外观偏移,难以获得密集真实标签,传统RGB训练数据仅提供有限监督。本文通过在现有对应基准上引入无传感器依赖的跨光谱调制协议与强度输入投影,提出一种模拟物理合理辐射差异的合成跨光谱对应基准。在多个现代密集对应骨干网络上评估表明,采用统一跨光谱协议训练后,在严重光谱失配条件下表现显著提升,同时保持对标准RGB基准的性能。消融实验显示,视图相关通道选择与非线性辐射变换提供互补鲁棒性,表明现有模型的主要瓶颈并非结构匹配能力,而是训练分布与目标图像对光谱特性间的不匹配。对异构医疗光谱采集系统的定性评估证实,所提训练数据增强协议具有实际意义,可作为高光谱成像工作流中空间一致融合的支撑技术。
原文摘要 · Abstract (English)
Precise dense correspondence is a fundamental prerequisite for multimodal spectral imaging systems that fuse disparate wavelength ranges for subsequent analysis in medical and scientific imaging. Corresponding image points are often observed with non-overlapping spectral sensitivities, leading to wavelength-dependent contrast changes, intensity inversions, and appearance shifts for which dense ground truth is difficult to obtain and conventional RGB-based training data provides only limited supervision. We address this data gap by introducing a sensor-agnostic cross-spectral modulation protocol on established correspondence benchmarks with intensity input projection, and by proposing a synthetic cross-spectral correspondence benchmark simulating physically plausible radiometric differences. Evaluation on several modern dense correspondence backbones trained with our unified cross-spectral protocol showed substantial improvements under severe spectral mismatch while maintaining performance on standard RGB benchmarks. Ablation experiments show that view-dependent channel selection and nonlinear radiometric transformations provide complementary robustness, indicating that the primary limitation of existing models is not their structural matching capacity but the mismatch between training distribution and spectral characteristics of the target image pair. Qualitative evaluations on heterogeneous medical spectral acquisition systems demonstrate the practical relevance of the proposed training data augmentation protocol as an enabler for spatially coherent spectral fusion in HSI workflows.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。