用少量光谱数据+扫描电镜图像，实现矿物精准分割
Mineral segmentation using electron microscope images and spectral sampling through multimodal graph neural networks
- 构建多模态图神经网络融合扫描电镜与光谱数据
- 仅需1%像素的光谱数据即可实现高精度分割
- 适合需要快速分析矿物成分的地质或材料研究
我们提出一种基于图神经网络的新型方法，通过融合多模态扫描电子显微镜（SEM）图像进行矿物分割。通常，使用SEM获取的背散射电子（BSE）图像信息不足，难以区分矿物。因此，常辅以逐点能量色散X射线光谱（EDS）测量，虽化学成分准确但耗时。这促使我们采用稀疏的EDS数据结合BSE图像进行分割。由于光谱数据无结构，传统图像融合方法不适用。我们采用图神经网络实现两类模态融合，并同步完成矿物相分割。结果表明，仅提供1% BSE像素的EDS数据即可实现高精度分割，显著提升矿物样品分析速度。该数据融合流程具备通用性，可拓展至其他包含图像与点测量数据的领域。
原文摘要 · Abstract (English)
We propose a novel Graph Neural Network-based method for segmentation based on data fusion of multimodal Scanning Electron Microscope (SEM) images. In most cases, Backscattered Electron (BSE) images obtained using SEM do not contain sufficient information for mineral segmentation. Therefore, imaging is often complemented with point-wise Energy-Dispersive X-ray Spectroscopy (EDS) spectral measurements that provide highly accurate information about the chemical composition but that are time-consuming to acquire. This motivates the use of sparse spectral data in conjunction with BSE images for mineral segmentation. The unstructured nature of the spectral data makes most traditional image fusion techniques unsuitable for BSE-EDS fusion. We propose using graph neural networks to fuse the two modalities and segment the mineral phases simultaneously. Our results demonstrate that providing EDS data for as few as 1% of BSE pixels produces accurate segmentation, enabling rapid analysis of mineral samples. The proposed data fusion pipeline is versatile and can be adapted to other domains that involve image data and point-wise measurements.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。