对比大模型与大脑活动，发现标点影响语义理解。
Aligning Brain Activity with Advanced Transformer Models: Exploring the Role of Punctuation in Semantic Processing
- 用神经数据测试四种Transformer模型的语义对齐度。
- RoBERTa与大脑活动最吻合，去标点后BERT表现反而更好。
- 揭示标点在人类语义处理中的关键作用，适合认知神经科学读者。
本研究考察了神经活动与先进Transformer模型之间的对应关系，强调标点在文本理解中的语义重要性。采用Toneva和Wehbe提出的方法，评估了四种先进Transformer模型（RoBERTa、DistilBERT、ALBERT、ELECTRA）与神经活动数据的一致性。结果表明，RoBERTa在与神经活动对齐方面表现最佳，优于BERT。此外，我们研究了去除标点对模型性能及神经对齐的影响，发现去除标点后BERT的准确率有所提升。该研究有助于理解神经网络如何表征语言，并揭示标点在人脑语义处理中的作用。
原文摘要 · Abstract (English)
This research examines the congruence between neural activity and advanced transformer models, emphasizing the semantic significance of punctuation in text understanding. Utilizing an innovative approach originally proposed by Toneva and Wehbe, we evaluate four advanced transformer models RoBERTa, DistiliBERT, ALBERT, and ELECTRA against neural activity data. Our findings indicate that RoBERTa exhibits the closest alignment with neural activity, surpassing BERT in accuracy. Furthermore, we investigate the impact of punctuation removal on model performance and neural alignment, revealing that BERT's accuracy enhances in the absence of punctuation. This study contributes to the comprehension of how neural networks represent language and the influence of punctuation on semantic processing within the human brain.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。