用非侵入式脑电数据实时翻译自然句子,准确率接近手术植入设备。
Accurate Decoding of Natural Sentences from Non-Invasive Brain Recordings

- 用深度学习替代人工设计流程,结合大语言模型提取语义特征。
- 平均词错误率39%,最佳用户半数句子仅错1词。
- 数据量越大精度越高,有望缩小与侵入式技术差距。
恢复脑损伤后丧失言语或运动能力者的交流能力是重大挑战。尽管颅内植入物已实现高性能脑机接口,非侵入式方法仍落后。本文提出Brain2Qwerty v2,仅基于实时脑磁图(MEG)记录解码自然句子。通过收集9名受试者每人10小时、共22,000句打字文本,模型融合字符、词和句子级表示,平均词错误率(WER)达39%。最优受试者中,超过一半句子仅含一个词错误。关键的是,解码准确率随数据量呈对数线性提升,表明通过数据扩展可部分缩小与侵入式方法的差距。人工智能通过三项关键机制实现此性能:以深度学习替代人工事件检测流程;微调大语言模型提取语义表征;部署AI代理自动优化解码管线。结果表明,非侵入式脑到文本解码已达到此前仅见于手术植入设备的水平,为安全高效的脑机接口开辟新路径。
原文摘要 · Abstract (English)
Restoring communication for people who have lost the ability to speak or move after a brain injury is a major challenge. While intracranial implants now enable high-performing brain-computer-interfaces, non-invasive alternatives are still lagging behind. Here, we present Brain2Qwerty v2, a model that can decode the production of natural sentences solely from real-time magnetoencephalography (MEG) recordings. By collecting 22,000 sentences typed by nine subjects, each recorded for 10 hours, our model leverages character, word and sentence-level representations to achieve an average word error rate (WER) of 39%. For our best participant, the model accurately decodes half of the sentences with one word error or less. Critically, decoding accuracy log-linearly improves with data volume, suggesting that the performance gap with intracranial approaches could be partially bridged through data scaling. We show that AI enables this performance in three main ways: the substitution of hand-crafted pipelines for event detection with deep learning, the finetuning of large language models to extract semantic representations, and the deployment of AI agents to iteratively refine our decoding pipeline via automated code development. Together, these results show that non-invasive brain-to-text decoding starts to operate at a level of accuracy previously thought exclusive to surgical implants, opening a path toward safe and efficient brain-computer-interfaces.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。