首个面向中文非侵入式语音脑机接口的多模态解码研究
Magnetoencephalography (MEG) Based Non-Invasive Chinese Speech Decoding
- 提出多模态辅助解码算法，融合文本与声学信息
- 构建首个中文脑磁图语音数据集，支持非侵入解码
- 为失语症患者提供新沟通可能，适合脑机接口研究者
作为脑机接口（BCI）的新兴范式，语音脑机接口有望直接反映听觉感知与思维，为失语症患者提供有前景的交流替代方案。中文是全球使用最广泛的语言之一，但针对中文语音脑机接口的研究极为有限。本文报告了一个基于脑磁图（MEG）的中文非侵入式语音脑机接口文本-脑磁图数据集，并提出一种多模态辅助语音解码（MASD）算法，以捕捉说话过程中脑信号中嵌入的文本与声学信息。实验结果验证了该文本-MEG数据集和所提MASD算法的有效性。据我们所知，这是首个关于非侵入式语音脑机接口的模态辅助解码研究。
原文摘要 · Abstract (English)
As an emerging paradigm of brain-computer interfaces (BCIs), speech BCI has the potential to directly reflect auditory perception and thoughts, offering a promising communication alternative for patients with aphasia. Chinese is one of the most widely spoken languages in the world, whereas there is very limited research on speech BCIs for Chinese language. This paper reports a text-magnetoencephalography (MEG) dataset for non-invasive Chinese speech BCIs. It also proposes a multi-modality assisted speech decoding (MASD) algorithm to capture both text and acoustic information embedded in brain signals during speech activities. Experiment results demonstrated the effectiveness of both our text-MEG dataset and our proposed MASD algorithm. To our knowledge, this is the first study on modality-assisted decoding for non-invasive speech BCIs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。