通过风格重建提升脑电图像解码精度，更真实还原视觉体验。
BrainDecoder: Style-Based Visual Decoding of EEG Signals
- 在CLIP空间中分别学习图像与文本特征，实现风格与语义分离解码
- 在Brain2Image数据集上显著提升风格还原效果，达新基准性能
- 适合脑机接口、神经科学和生成模型研究者参考
从脑电图（EEG）中解码视觉刺激的神经表征为理解大脑活动与认知提供了重要视角。深度学习的进步显著推动了视觉解码的发展，主要集中于恢复视觉刺激的语义内容。本文提出一种新型视觉解码流程，不仅恢复内容，还特别关注图像颜色、纹理等风格信息的重建。不同于以往方法，该“风格导向”策略在图像与文本的CLIP空间中分别学习，提升了从EEG信号中提取细粒度信息的能力。我们采用更简单的描述文本进行对齐，发现其效果优于先前方法。定量与定性评估均表明，本方法在保留视觉风格方面表现更优，并能从神经信号中提取更丰富的语义信息。尤其在流行的Brain2Image数据集上取得显著提升，达到新的最优水平。
原文摘要 · Abstract (English)
Decoding neural representations of visual stimuli from electroencephalography (EEG) offers valuable insights into brain activity and cognition. Recent advancements in deep learning have significantly enhanced the field of visual decoding of EEG, primarily focusing on reconstructing the semantic content of visual stimuli. In this paper, we present a novel visual decoding pipeline that, in addition to recovering the content, emphasizes the reconstruction of the style, such as color and texture, of images viewed by the subject. Unlike previous methods, this ``style-based'' approach learns in the CLIP spaces of image and text separately, facilitating a more nuanced extraction of information from EEG signals. We also use captions for text alignment simpler than previously employed, which we find work better. Both quantitative and qualitative evaluations show that our method better preserves the style of visual stimuli and extracts more fine-grained semantic information from neural signals. Notably, it achieves significant improvements in quantitative results and sets a new state-of-the-art on the popular Brain2Image dataset.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。