大模型与人脑在处理语言时遵循相似的计算路径。
Scaling and context steer LLMs along the same computational path as the human brain
- 对比22个不同规模的大模型与人脑听觉信号，发现层间激活顺序一致。
- 小模型和短上下文下对齐效果弱，模型越大、上下文越长对齐越强。
- 适用于研究神经网络与大脑计算机制的交叉领域学者。
近期研究显示，大型语言模型（LLMs）学习到的表征与人脑部分对齐。然而，这种对齐是否源于相似的计算过程仍不清楚。本研究通过分析参与者聆听10小时有声书时的时序脑电信号，并结合涵盖22个不同规模和架构类型大模型的基准测试，发现LLMs与人脑生成表征的顺序高度一致：初始层的激活更接近早期脑响应，深层激活更接近后期脑响应。该对齐现象在Transformer与循环架构中均成立，但依赖于模型规模和上下文长度。结果揭示了生物与人工神经网络在计算序列上的部分收敛机制。
原文摘要 · Abstract (English)
Recent studies suggest that the representations learned by large language models (LLMs) are partially aligned to those of the human brain. However, whether and why this alignment score arises from a similar sequence of computations remains elusive. In this study, we explore this question by examining temporally-resolved brain signals of participants listening to 10 hours of an audiobook. We study these neural dynamics jointly with a benchmark encompassing 22 LLMs varying in size and architecture type. Our analyses confirm that LLMs and the brain generate representations in a similar order: specifically, activations in the initial layers of LLMs tend to best align with early brain responses, while the deeper layers of LLMs tend to best align with later brain responses. This brain-LLM alignment is consistent across transformers and recurrent architectures. However, its emergence depends on both model size and context length. Overall, this study sheds light on the sequential nature of computations and the factors underlying the partial convergence between biological and artificial neural networks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。