用深度学习提升脑电与磁脑信号的解码精度,探索内言语识别难题
Decoding non-invasive brain activity with novel deep-learning approaches
- 采用卷积与Transformer模型处理脑电/磁数据,应对个体间差异挑战
- 在视觉刺激解码中实现更高准确率,但内言语解码仍面临巨大困难
- 提出新方法提升群体解码性能,适合脑机接口与神经科学研究者
本论文聚焦非侵入式脑电图(EEG)与脑磁图(MEG)信号的建模与解码,旨在揭示视觉刺激或内言语状态下大脑活动机制,并提升解码性能。研究分为方法与实验两部分。方法上,提出基于深度学习的新模型,包括个体水平的线性解码、处理跨被试差异的群体解码方法,以及结合卷积与Transformer架构的MEG信号预测模型;其中,Transformer模型生成的信号更贴近真实脑活动,显著提升建模可靠性。实验上,构建包含高样本量内言语的EEG、MEG及初步光泵磁力计(OPM)数据集,以探索不同内言语类型并提高解码能力。然而,解码结果总体不理想,凸显内言语解码的高难度。
原文摘要 · Abstract (English)
This thesis delves into the world of non-invasive electrophysiological brain signals like electroencephalography (EEG) and magnetoencephalography (MEG), focusing on modelling and decoding such data. The research aims to investigate what happens in the brain when we perceive visual stimuli or engage in covert speech (inner speech) and enhance the decoding performance of such stimuli. The thesis is divided into two main sections, methodological and experimental work. A central concern in both sections is the large variability present in electrophysiological recordings, whether it be within-subject or between-subject variability, and to a certain extent between-dataset variability. In the methodological sections, we explore the potential of deep learning for brain decoding. We present advancements in decoding visual stimuli using linear models at the individual subject level. We then explore how deep learning techniques can be employed for group decoding, introducing new methods to deal with between-subject variability. Finally, we also explores novel forecasting models of MEG data based on convolutional and Transformer-based architectures. In particular, Transformer-based models demonstrate superior capabilities in generating signals that closely match real brain data, thereby enhancing the accuracy and reliability of modelling the brain's electrophysiology. In the experimental section, we present a unique dataset containing high-trial inner speech EEG, MEG, and preliminary optically pumped magnetometer (OPM) data. Our aim is to investigate different types of inner speech and push decoding performance by collecting a high number of trials and sessions from a few participants. However, the decoding results are found to be mostly negative, underscoring the difficulty of decoding inner speech.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。