发现语言模型训练中几何结构自组织，提升与人脑语言区的对齐。
Training-Driven Representational Geometry Modularization Predicts Brain Alignment in Language Models
- 通过追踪熵和曲率变化，发现模型层形成高低复杂度模块。
- 低复杂度模块预测人脑语言活动更准，尤其在颞叶区域。
- 模型规模越大，几何平滑越有助于模拟人脑语言处理机制。
大型语言模型（LLMs）如何与人类语言神经表征和计算过程对齐，是认知科学的核心问题。我们利用表示几何作为机制视角，追踪了Pythia（70M-1B）模型在训练过程中的熵、曲率及fMRI编码分数。发现层间出现几何模块化现象，即模型层自我组织为稳定且低复杂度与高复杂度的簇。其中，低复杂度模块表现出更低的熵与曲率，持续更准确地预测人类语言网络活动。这种对齐呈现出异质的空间-时间轨迹：在颞叶区域（AntTemp, PostTemp）快速且稳定，而在额叶区域（IFG, IFGorb）延迟且动态。关键的是，在控制训练进度后，曲率降低仍为模型-大脑对齐的稳健预测因子，且该效应随模型规模增大而增强。结果揭示了训练驱动的几何重组与时间-额叶功能特化的关联，表明表示平滑有助于实现类神经语言处理。
原文摘要 · Abstract (English)
How large language models (LLMs) align with the neural representation and computation of human language is a central question in cognitive science. Using representational geometry as a mechanistic lens, we addressed this by tracking entropy, curvature, and fMRI encoding scores throughout Pythia (70M-1B) training. We identified a geometric modularization where layers self-organize into stable low- and high-complexity clusters. The low-complexity module, characterized by reduced entropy and curvature, consistently better predicted human language network activity. This alignment followed heterogeneous spatial-temporal trajectories: rapid and stable in temporal regions (AntTemp, PostTemp), but delayed and dynamic in frontal areas (IFG, IFGorb). Crucially, reduced curvature remained a robust predictor of model-brain alignment even after controlling for training progress, an effect that strengthened with model scale. These results links training-driven geometric reorganization to temporal-frontal functional specialization, suggesting that representational smoothing facilitates neural-like linguistic processing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。