用RGB图像重建高光谱数据，提升手术成像精度
RGB to Hyperspectral: Spectral Reconstruction for Enhanced Surgical Imaging
- 结合CNN与Transformer模型，融合空间信息预测光谱
- Transformer在RMSE、SAM等指标上优于传统方法
- 可支持术中实时决策，适合临床手术场景
本研究探索从RGB图像重建高光谱信号以增强手术成像，使用公开的猪手术数据集HeiPorSPECTRAL及自建神经外科数据集。通过多种基于卷积神经网络（CNN）和Transformer的架构，在多维度指标下进行评估。Transformer模型在RMSE、SAM、PSNR和SSIM方面表现更优，能有效整合空间信息，准确预测涵盖可见光及扩展光谱范围的光谱曲线。定性分析显示其具备预测关键光谱特征的能力，有助于术中决策。同时，通过平均绝对误差（MAE）揭示了同时捕捉可见光与扩展光谱范围的挑战。研究为手术应用中的高光谱重建开辟新方向，具有实时临床应用潜力。
原文摘要 · Abstract (English)
This study investigates the reconstruction of hyperspectral signatures from RGB data to enhance surgical imaging, utilizing the publicly available HeiPorSPECTRAL dataset from porcine surgery and an in-house neurosurgery dataset. Various architectures based on convolutional neural networks (CNNs) and transformer models are evaluated using comprehensive metrics. Transformer models exhibit superior performance in terms of RMSE, SAM, PSNR and SSIM by effectively integrating spatial information to predict accurate spectral profiles, encompassing both visible and extended spectral ranges. Qualitative assessments demonstrate the capability to predict spectral profiles critical for informed surgical decision-making during procedures. Challenges associated with capturing both the visible and extended hyperspectral ranges are highlighted using the MAE, emphasizing the complexities involved. The findings open up the new research direction of hyperspectral reconstruction for surgical applications and clinical use cases in real-time surgical environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。