提出跨维度融合网络,提升病理显微高光谱图像分割精度。
Omni-Fusion of Spatial and Spectral for Hyperspectral Image Segmentation
- 通过双向注意力机制融合空间与光谱特征
- 在两个数据集上实现超5.73%的DSC提升
- 适合医学图像分析与高光谱处理研究者
医学高光谱成像(MHSI)在计算病理学中展现出增强疾病诊断的潜力,能提供丰富的光谱信息以识别组织的细微生化特性。然而,由于高维性和光谱冗余性,有效融合空间与光谱信息仍具挑战。为此,我们提出一种新型空间-光谱全维度融合网络Omni-Fuse。该方法引入多种跨维度特征融合操作:包括通过双向注意力机制优化空间与光谱特征的交叉增强模块;基于光谱引导的空间查询选择机制,选取最相关的空间特征作为查询;以及两阶段动态引导的跨维度解码器,聚焦于选定的查询。尽管包含多个注意力模块,Omni-Fuse仍保持高效。在两个显微高光谱图像数据集上的实验表明,相比当前最优方法,本方法显著提升分割性能,平均交并比(DSC)提升超过5.73%。代码已开源:https://github.com/DeepMed-Lab-ECNU/Omni-Fuse。
原文摘要 · Abstract (English)
Medical Hyperspectral Imaging (MHSI) has emerged as a promising tool for enhanced disease diagnosis, particularly in computational pathology, offering rich spectral information that aids in identifying subtle biochemical properties of tissues. Despite these advantages, effectively fusing both spatial-dimensional and spectral-dimensional information from MHSIs remains challenging due to its high dimensionality and spectral redundancy inherent characteristics. To solve the above challenges, we propose a novel spatial-spectral omni-fusion network for hyperspectral image segmentation, named as Omni-Fuse. Here, we introduce abundant cross-dimensional feature fusion operations, including a cross-dimensional enhancement module that refines both spatial and spectral features through bidirectional attention mechanisms, a spectral-guided spatial query selection to select the most spectral-related spatial feature as the query, and a two-stage cross-dimensional decoder which dynamically guide the model to focus on the selected spatial query. Despite of numerous attention blocks, Omni-Fuse remains efficient in execution. Experiments on two microscopic hyperspectral image datasets show that our approach can significantly improve the segmentation performance compared with the state-of-the-art methods, with over 5.73 percent improvement in DSC. Code available at: https://github.com/DeepMed-Lab-ECNU/Omni-Fuse.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。