用脑电图生成高精度脑部影像,突破传统方法的性能与效率瓶颈。
Spec2VolCAMU-Net: A Spectrogram-to-Volume Model for EEG-to-fMRI Reconstruction based on Multi-directional Time-Frequency Convolutional Attention Encoder and Vision-Mamba U-Net
- 多方向时频卷积注意力编码器提取脑电信号深层特征
- 在三个公开数据集上SSIM最高达0.788,提升超14%
- 轻量化设计适合临床实时应用,代码开源
高分辨率功能磁共振成像(fMRI)对脑活动映射至关重要,但成本高且难实施。若能从广泛可用的头皮脑电图(EEG)直接生成类似体积,先进神经成像将更易获取。现有方法依赖普通卷积网络,难以捕捉跨通道时频特征,或使用重型变换器/GAN解码器,导致内存与稳定性问题。本文提出Spec2VolCAMU-Net,采用多方向时频卷积注意力编码器实现高效特征提取,结合视觉Mamba U-Net解码器,利用线性时间状态空间块进行长程空间建模。模型以混合SSI-MSE损失端到端训练,目标为提升单体积重建的空间保真度,为生成时序连贯的fMRI序列奠定基础。在三个公开基准上达到新最佳性能:NODDI数据集SSIM为0.693,比前人提升14.5%;Oddball为0.725(+14.9%);CN-EPFL为0.788(+16.9%)。同时在PSNR上表现优异,尤其在CN-EPFL上较前人提升4.6%,重建质量更均衡。模型轻量高效,适用于临床与科研中的实时场景。代码已开源。
原文摘要 · Abstract (English)
High-resolution functional magnetic resonance imaging (fMRI) is essential for mapping human brain activity; however, it remains costly and logistically challenging. If comparable volumes could be generated directly from widely available scalp electroencephalography (EEG), advanced neuroimaging would become significantly more accessible. Existing EEG-to-fMRI generators rely on plain Convolutional Neural Networks (CNNs) that fail to capture cross-channel time-frequency cues or on heavy transformer/Generative Adversarial Network (GAN) decoders that strain memory and stability. To address these limitations, we propose Spec2VolCAMU-Net, a lightweight architecture featuring a Multi-directional Time-Frequency Convolutional Attention Encoder for rich feature extraction and a Vision-Mamba U-Net decoder that uses linear-time state-space blocks for efficient long-range spatial modelling. We frame the goal of this work as establishing a new state of the art in the spatial fidelity of single-volume reconstruction, a foundational prerequisite for the ultimate aim of generating temporally coherent fMRI time series. Trained end-to-end with a hybrid SSI-MSE loss, Spec2VolCAMU-Net achieves state-of-the-art fidelity on three public benchmarks, recording Structural Similarity Index (SSIM) of 0.693 on NODDI, 0.725 on Oddball and 0.788 on CN-EPFL, representing improvements of 14.5%, 14.9%, and 16.9% respectively over previous best SSIM scores. Furthermore, it achieves competitive Signal-to-Noise Ratio (PSNR) scores, particularly excelling on the CN-EPFL dataset with a 4.6% improvement over the previous best PSNR, thus striking a better balance in reconstruction quality. The proposed model is lightweight and efficient, making it suitable for real-time applications in clinical and research settings. The code is available at https://github.com/hdy6438/Spec2VolCAMU-Net.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。