提出新基准,证明非侵入式脑电可无教师强制实现文本解码。
Is EEG-to-Text Feasible in Real-World Scenarios? An In-Depth Analysis Using a Neuropsychology-Inspired Benchmark

- 设计基于神经心理学的评估范式,解决脑电信号不稳定性问题。
- 在128通道高密度脑电下,首次实现无需教师强制的稳定文本生成。
- 开源数据集COFETT,支持真实场景下的模型评估与应用开发。
将脑信号转为文本可恢复重度瘫痪者的沟通能力,但现有系统多依赖侵入式皮层脑电图(ECoG)。非侵入式脑电图(EEG)虽具潜力,但当前EEG-to-text(EEG2Text)模型普遍依赖教师强制评估,脱离该机制则无法生成有效文本,阻碍其在真实场景中的应用。这一现象引发关于EEG是否蕴含可解码语言信息的争议。本文通过神经心理学启发的范式发现,现有基准忽视了脑电信号的不稳定性,导致推理结果混淆。实验表明,摒弃教师强制后仍可实现有效的文本解码。为此,我们构建了128通道高密度脑电帽采集的脑电-文本语料库COFETT,作为专用基准,显著提升模型性能区分能力,并支持鲁棒的无教师强制评估,为实际应用开辟路径。数据集已开源:https://github.com/baoyudu/COFETT。
原文摘要 · Abstract (English)
Translating brain signals into text could restore communication for people with severe paralysis, yet practically usable systems to date rely on invasive electrocorticography (ECoG). Electroencephalography (EEG) offers a non-invasive alternative, and EEG-to-text (EEG2Text) has been widely explored. Interestingly, however, EEG2Text models generally rely on teacher-forcing evaluation; without it, they fail to generate meaningful decoding. This reliance prevents EEG2Text from being applied in real-world, non-academic settings. This has fueled numerous debates about whether EEG2Text is a meaningful direction, by extension, and whether EEG truly contains decodable linguistic information. Here, using a neuropsychology-informed paradigm, we find that existing EEG2Text benchmarks have neglected EEG instability, a flaw that has confounded inference and sparked debate. Our experiments furnish key evidence for the feasibility of teacher-forcing-free EEG2Text decoding. Accordingly, we assemble the Corpus OF Eeg-To-Text (COFETT) using a 128-channel high-density EEG cap, providing a benchmark dedicated to evaluating EEG2Text models. In comparisons with multiple existing benchmarks, COFETT achieves SOTA ability to distinguish among model performances and enables robust, teacher-forcing-free evaluation, thereby opening a path toward practical EEG2Text applications. COFETT is open sourced in https://github.com/baoyudu/COFETT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。