用脑电数据直接还原说话内容的语义，仅需30分钟训练。
Neuro2Semantic: A Transfer Learning Framework for Semantic Reconstruction of Continuous Language from Human Intracranial EEG
- 分两阶段：先对齐脑电与文本嵌入，再生成自然语言。
- 30分钟数据即超越当前最优方法，低数据下表现优异。
- 适合脑机接口、神经解码研究者，推动实时语言重建。
从神经信号中解码连续语言仍是神经科学与人工智能交叉领域的重大挑战。我们提出Neuro2Semantic框架，从皮层内脑电（iEEG）记录中重构感知语音的语义内容。该方法分为两个阶段：首先使用基于LSTM的适配器将神经信号与预训练文本嵌入对齐；其次通过校正模块直接从对齐嵌入生成连续自然文本。该灵活方法克服了以往解码方法的局限，实现无约束文本生成。Neuro2Semantic仅需30分钟神经数据即可达到出色性能，在低数据设置下优于近期最先进方法。这些结果凸显其在脑机接口与神经解码技术中的实际应用潜力。
原文摘要 · Abstract (English)
Decoding continuous language from neural signals remains a significant challenge in the intersection of neuroscience and artificial intelligence. We introduce Neuro2Semantic, a novel framework that reconstructs the semantic content of perceived speech from intracranial EEG (iEEG) recordings. Our approach consists of two phases: first, an LSTM-based adapter aligns neural signals with pre-trained text embeddings; second, a corrector module generates continuous, natural text directly from these aligned embeddings. This flexible method overcomes the limitations of previous decoding approaches and enables unconstrained text generation. Neuro2Semantic achieves strong performance with as little as 30 minutes of neural data, outperforming a recent state-of-the-art method in low-data settings. These results highlight the potential for practical applications in brain-computer interfaces and neural decoding technologies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。