用预训练模型给脑电数据打情感标签，实现从脑信号解码情绪。
MEGnifying Emotion: Sentiment Analysis from Annotated Brain Data
- 用文本转情感模型为听书时的脑电数据打情感标签。
- 脑信号到情感的分类平衡准确率优于基线模型。
- 适合脑机接口、情绪计算研究者参考。
从脑活动解码情绪可深化对人类体验的理解。现有数据集虽将脑数据与语音及语音转录文本对齐，但缺乏情感标注。为此，我们探索使用预训练的Text-to-Sentiment模型，为参与听有声书时采集的非侵入式脑磁图（MEG）数据打情感标签。在标注文本后，通过文本与音频的强制对齐，将情感标签精确对齐至脑记录。由此可直接训练脑信号到情感的模型。实验表明，相比基线，脑信号到情感的分类平衡准确率有所提升，验证了该方法作为利用现有MEG数据直接从脑信号解码情感的可行性证明。
原文摘要 · Abstract (English)
Decoding emotion from brain activity could unlock a deeper understanding of the human experience. While a number of existing datasets align brain data with speech and with speech transcripts, no datasets have annotated brain data with sentiment. To bridge this gap, we explore the use of pre-trained Text-to-Sentiment models to annotate non invasive brain recordings, acquired using magnetoencephalography (MEG), while participants listened to audiobooks. Having annotated the text, we employ force-alignment of the text and audio to align our sentiment labels with the brain recordings. It is straightforward then to train Brainto-Sentiment models on these data. Experimental results show an improvement in balanced accuracy for Brain-to-Sentiment compared to baseline, supporting the proposed approach as a proof-of-concept for leveraging existing MEG datasets and learning to decode sentiment directly from the brain.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。