通过共享空间对齐实现跨人脑语义解码,提升神经信号泛化能力。
Cross-Subject Semantic Decoding with Shared-Space Alignment for Generalized Neural Representation Learning

- 将多人的脑电数据映射到共享潜在空间,统一神经响应表示
- 在未见受试者上直接使用预训练解码器,性能下降更小
- 适合需要跨个体泛化的脑机接口与神经解码研究
在侵入式神经记录中,跨受试者泛化仍具挑战,因电极配置、解剖结构及神经信号模式存在显著个体差异。本文提出一种跨受试者语义解码框架,将多位受试者在自然语言理解过程中的皮层电图(ECoG)数据映射至共享潜在空间,并学习从对齐后的神经表征到上下文嵌入的映射关系。具体地,利用共享响应模型估计共享空间,训练解码器以预测上下文语义嵌入。对于新受试者,仅需估计其个性化的投影矩阵,即可直接应用预训练解码器,无需重新训练。实验表明,该框架在多种评估设置下均优于基线方法,且从源受试者到未见受试者的性能下降更小,验证了其跨受试者泛化能力的提升。结果表明,将神经活动对齐至共享空间并结合语义嵌入解码,可有效减少个体差异,同时保留与刺激相关的共享表征,是提升跨个体泛化的重要策略。
原文摘要 · Abstract (English)
Generalizing across subjects remains challenging in invasive neural recordings because electrode configurations, anatomical structures, and neural signal patterns vary substantially across individuals. To investigate such inter-subject variability, we propose a cross-subject semantic decoding framework that aligns neural responses to speech perception from multiple subjects into a shared latent space and learns a mapping from the aligned neural representations to contextual embeddings. More specifically, using electrocorticography data collected during natural language comprehension, we estimate the shared space using the shared response model and train a decoder to predict contextual semantic embeddings from projected neural responses. For a held-out subject, we estimate a subject-specific projection into the predefined shared space, and directly apply the pretrained decoder without any retraining. Experimental results demonstrate that the proposed framework consistently outperforms baseline methods across evaluation settings and exhibits a reduced performance drop from source subject to held-out subject testing, indicating improved cross-subject generalization. These results suggest that aligning neural activity into a shared latent space, while decoding in a semantic embedding space, provides an effective strategy for improving cross-subject generalization by reducing subject-specific differences in neural responses while effectively capturing shared stimulus-related representations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。