用新模型将脑电图信号转为文本，准确率显著提升。
EEG-to-Text Translation: A Model for Deciphering Human Brain Activity
- 结合双向LSTM与预训练Transformer，捕捉脑电信号时序特征并生成文本。
- ROUGE-1达38.00%，比现有模型高3%-9%，字错率降低4%以上。
- 适合脑机接口、神经解码研究者使用，代码开源可复现。
随着Gemini、GPT等大语言模型的快速发展，实现人脑与语言处理之间的桥梁成为研究热点。为应对这一挑战，研究人员已开发多种将脑电图（EEG）信号解码为文本的模型，但性能仍有限。为此，本文提出R1 Translator模型，通过双向LSTM编码器与预训练Transformer解码器结合，利用EEG特征生成高质量文本。该模型先用LSTM提取序列依赖关系，再由Transformer解码生成文本。在多个指标上，R1均优于T5和Brain Translator：ROUGE-1得分为38.00%（精度），较T5（34.89%）提升9%，较Brain（35.69%）提升3%；ROUGE-L F1达32.51%，优于T5（29.67%）3%，Brain（30.38%）2%；CER为0.5795，低于T5（0.5917）2%，Brain（0.6001）4%；WER为0.7280，优于T5（0.7610）4.3%，Brain（0.7553）3.6%。代码已开源：https://github.com/Mmurrad/EEG-To-text。
原文摘要 · Abstract (English)
With the rapid advancement of large language models like Gemini, GPT, and others, bridging the gap between the human brain and language processing has become an important area of focus. To address this challenge, researchers have developed various models to decode EEG signals into text. However, these models still face significant performance limitations. To overcome these shortcomings, we propose a new model, R1 Translator, which aims to improve the performance of EEG-to-text decoding. The R1 Translator model combines a bidirectional LSTM encoder with a pretrained transformer-based decoder, utilizing EEG features to produce high-quality text outputs. The model processes EEG embeddings through the LSTM to capture sequential dependencies, which are then fed into the transformer decoder for effective text generation. The R1 Translator excels in ROUGE metrics, outperforming both T5 (previous research) and Brain Translator. Specifically, R1 achieves a ROUGE-1 score of 38.00% (P), which is up to 9% higher than T5 (34.89%) and 3% better than Brain (35.69%). It also leads in ROUGE-L, with a F1 score of 32.51%, outperforming T5 by 3% (29.67%) and Brain by 2% (30.38%). In terms of CER, R1 achieves a CER of 0.5795, which is 2% lower than T5 (0.5917) and 4% lower than Brain (0.6001). Additionally, R1 performs better in WER with a score of 0.7280, outperforming T5 by 4.3% (0.7610) and Brain by 3.6% (0.7553). Code is available at https://github.com/Mmurrad/EEG-To-text.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。