让脑电转文本更可信:用可解释表示减少幻觉
Learning Interpretable Representations Leads to Semantically Faithful EEG-to-Text Generation
- 将脑电转文本重构为语义摘要,避免逐字还原
- 在ZuCo数据集上生成流畅且与脑电信号一致的句子
- 支持情绪、关系等零样本分类,适合脑机接口研究者
预训练生成模型为非侵入性脑记录解码开辟了新路径,能合成逼真的文本和图像。然而,这些输出是否真实反映大脑语义激活仍存疑,可能只是生成模型的幻觉。本文聚焦脑电到文本的解码,从后验坍缩角度解决幻觉问题。考虑到脑电与文本间的信息容量差异,我们将任务重新定义为对核心语义的语义摘要,而非原始刺激文本的逐字重建。为此,提出生成式语言检视模型(GLIM),强调学习具有信息量且可解释的脑电信号表征,以提升在异构小样本数据下的语义一致性。在公开的ZuCo数据集上的实验表明,GLIM无需教师强制即可持续生成流畅且与脑电信号一致的句子。此外,其评估方式超越文本相似度,支持脑电-文本检索和跨情感类别、关系类型、语料主题的零样本语义分类,具备更强鲁棒性。整体架构与评估协议为生成式脑解码的可靠与可扩展基准测试奠定基础。
原文摘要 · Abstract (English)
Pretrained generative models have opened new frontiers in brain decoding by enabling the synthesis of realistic texts and images from non-invasive brain recordings. However, the reliability of such outputs remains questionable--whether they truly reflect semantic activation in the brain, or are merely hallucinated by the powerful generative models. In this paper, we focus on EEG-to-text decoding and address its hallucination issue through the lens of posterior collapse. Acknowledging the underlying mismatch in information capacity between EEG and text, we reframe the decoding task as semantic summarization of core meanings rather than previously verbatim reconstruction of stimulus texts. To this end, we propose the Generative Language Inspection Model (GLIM), which emphasizes learning informative and interpretable EEG representations to improve semantic grounding under heterogeneous and small-scale data conditions. Experiments on the public ZuCo dataset demonstrate that GLIM consistently generates fluent, EEG-grounded sentences without teacher forcing. Moreover, it supports more robust evaluation beyond text similarity, through EEG-text retrieval and zero-shot semantic classification across sentiment categories, relation types, and corpus topics. Together, our architecture and evaluation protocols lay the foundation for reliable and scalable benchmarking in generative brain decoding.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。