arXiv:2506.03214q-bio.NCcs.AI2025-06被引 2

跨语言跨主体脑信号解码框架,提升BCI通用性与语言公平性

A Pre-trained Framework for Multilingual Brain Decoding Using Non-invasive Recordings

  • 用预训练多语言模型构建统一语义空间,融合多模态脑数据
  • 159人4语言验证,跨语言/主体/模态解码性能显著提升
  • 特别增强小语种表现,推动脑机接口语言公平

基于脑电记录的脑-机接口(BCI)在临床康复和认知神经科学中具有广阔应用前景。然而,现有解码方法仍局限于单一语言、单一受试者和单一神经成像模态,限制了其临床适用性和泛化能力。本文提出一个联合多语言、多受试者、多模态的解码框架,将多样化的脑记录映射到由预训练多语言模型(PMM)定义的统一语义空间中,实现跨语言、跨受试者、跨模态的解码。该框架在来自159名参与者、涵盖四种语言的非侵入式脑记录上进行了验证。实验结果表明,其在多语言、多受试者、多模态设置下展现出强泛化能力。更重要的是,该框架能促进语言公平,对脑机接口中代表性不足的语言尤为关键。统一语义空间增强了跨语言映射,从而提升小语种的解码性能,推动语言公平。总体而言,该框架为脑解码建立了新范式,开辟了BCI更广泛应用的新路径。

原文摘要 · Abstract (English)

Brain-computer interfaces (BCIs) with speech decoding from brain recordings have broad application potential in fields such as clinical rehabilitation and cognitive neuroscience. However, current decoding methods remain limited to single-language, single-subject, and single neuroimaging modality settings, restricting their clinical applicability and generalizability. Here we propose a joint multilingual, multi-subject and multimodal decoding framework. It maps diverse brain recordings into a unified semantic space defined by a pre-trained multilingual model (PMM), enabling decoding across multiple languages, multiple subjects and multiple neuroimaging modalities. The proposed framework is validated using non-invasive brain recordings from 159 participants across four languages. Experimental results show that it exhibits strong generalization across multilingual, multi-subject, and multimodal settings. More importantly, the proposed framework can promote linguistic fairness, which is vital for underrepresented languages in BCI applications. The unified semantic space enables cross-lingual mapping enhancement, allowing the framework to boost the decoding performance of underrepresented languages, thereby promoting linguistic fairness. Overall, the proposed framework establishes a new potential paradigm for brain decoding, opening new paths for broader applications of BCI.

脑机接口多语言跨模态语言公平

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。