用EEG数据生成高分辨率fMRI图像,精度显著提升。
NT-ViT: Neural Transcoding Vision Transformers for EEG-to-fMRI Synthesis
- 通过域匹配模块对齐脑电与脑磁的潜在特征
- 在奇数刺激数据集上实现RMSE降低10倍、SSIM提升3.14倍
- 适合需要低成本脑成像的临床与神经科学研究
本文提出神经转码视觉变换器(NT-ViT),一种生成模型,可从同步采集的脑电图(EEG)数据中估计高分辨率功能磁共振成像(fMRI)样本。其核心在于域匹配(DM)子模块,能有效对齐EEG与fMRI体积的潜在表示,显著提升模型准确性和可靠性。相比以往方法在图像保真度和可重复性上的不足,NT-ViT通过保证方法完整性,实现了更高品质重建,在两个基准数据集上均显著超越现有最先进水平,例如在奇数刺激数据集上实现RMSE降低10倍、SSIM提升3.14倍。消融实验揭示了各组件对整体性能的贡献。该技术虽不能替代真实fMRI,但有望缓解高分辨率脑成像的时间与经济成本,助力神经疾病快速精准诊断。代码已公开于https://github.com/rom42pla/ntvit。
原文摘要 · Abstract (English)
This paper introduces the Neural Transcoding Vision Transformer (\modelname), a generative model designed to estimate high-resolution functional Magnetic Resonance Imaging (fMRI) samples from simultaneous Electroencephalography (EEG) data. A key feature of \modelname is its Domain Matching (DM) sub-module which effectively aligns the latent EEG representations with those of fMRI volumes, enhancing the model's accuracy and reliability. Unlike previous methods that tend to struggle with fidelity and reproducibility of images, \modelname addresses these challenges by ensuring methodological integrity and higher-quality reconstructions which we showcase through extensive evaluation on two benchmark datasets; \modelname outperforms the current state-of-the-art by a significant margin in both cases, e.g. achieving a $10\times$ reduction in RMSE and a $3.14\times$ increase in SSIM on the Oddball dataset. An ablation study also provides insights into the contribution of each component to the model's overall effectiveness. This development is critical in offering a new approach to lessen the time and financial constraints typically linked with high-resolution brain imaging, thereby aiding in the swift and precise diagnosis of neurological disorders. Although it is not a replacement for actual fMRI but rather a step towards making such imaging more accessible, we believe that it represents a pivotal advancement in clinical practice and neuroscience research. Code is available at \url{https://github.com/rom42pla/ntvit}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。