用小波变换和语义损失提升脑电图像重建的准确性
Category-aware EEG image generation based on wavelet transform and contrast semantic loss
- 结合小波变换与门控机制，从脑电信号中提取视觉特征
- 在THINGS-EEG数据集上达到43%单人分类准确率
- 提出新语义评分方法，更好评估生成图像的语义一致性
从脑电信号重构视觉刺激是实现脑机接口的关键步骤。本文提出一种基于Transformer的脑电信号编码器，融合离散小波变换（DWT）与门控机制，通过特征对齐和类别感知融合损失，从脑电信号中提取与视觉刺激相关的特征。随后，利用预训练的扩散模型将这些特征重建为视觉刺激。为验证模型有效性，我们在THINGS-EEG数据集上进行了脑电到图像生成与分类任务。针对语义层面量化分析的局限性，我们结合WordNet分类与语义相似性度量，提出一种新型语义评分，强调模型将神经活动转化为视觉表征的能力。实验结果表明，该模型显著提升了语义对齐效果与分类准确率，最高单人准确率达43%，优于现有先进方法。源代码与补充材料见https://github.com/zes0v0inn/DWT_EEG_Reconstruction/tree/main。
原文摘要 · Abstract (English)
Reconstructing visual stimuli from EEG signals is a crucial step in realizing brain-computer interfaces. In this paper, we propose a transformer-based EEG signal encoder integrating the Discrete Wavelet Transform (DWT) and the gating mechanism. Guided by the feature alignment and category-aware fusion losses, this encoder is used to extract features related to visual stimuli from EEG signals. Subsequently, with the aid of a pre-trained diffusion model, these features are reconstructed into visual stimuli. To verify the effectiveness of the model, we conducted EEG-to-image generation and classification tasks using the THINGS-EEG dataset. To address the limitations of quantitative analysis at the semantic level, we combined WordNet-based classification and semantic similarity metrics to propose a novel semantic-based score, emphasizing the ability of our model to transfer neural activities into visual representations. Experimental results show that our model significantly improves semantic alignment and classification accuracy, which achieves a maximum single-subject accuracy of 43\%, outperforming other state-of-the-art methods. The source code and supplementary material is available at https://github.com/zes0v0inn/DWT_EEG_Reconstruction/tree/main.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。