用预训练语言模型模拟人类认知发展，需警惕方法误区。
The potential -- and the pitfalls -- of using pre-trained language models as cognitive science theories
- 将模型性能映射为人类认知的假设需谨慎验证
- 模型架构与训练数据差异影响结论可信度
- 适合关注认知科学建模的跨学科研究者
许多研究评估了预训练语言模型（PLMs）与成人认知表现的一致性，涵盖多个认知领域。近期研究扩展至发展一致性：识别模型训练过程中性能提升与儿童思维发展相匹配的阶段。然而，将PLMs作为认知科学理论面临诸多挑战，包括模型架构差异、训练数据模态与规模不一，以及模型可解释性有限。本文总结将PLMs视为认知科学与发育科学模型的经验教训，回顾研究者用于将模型性能指标映射到人类表现所依赖的假设，揭示该方法在理解人类思维时的潜在陷阱，并提出使用PLMs作为可信认知与认知发展解释的评判标准。
原文摘要 · Abstract (English)
Many studies have evaluated the cognitive alignment of Pre-trained Language Models (PLMs), i.e., their correspondence to adult performance across a range of cognitive domains. Recently, the focus has expanded to the developmental alignment of these models: identifying phases during training where improvements in model performance track improvements in children's thinking over development. However, there are many challenges to the use of PLMs as cognitive science theories, including different architectures, different training data modalities and scales, and limited model interpretability. In this paper, we distill lessons learned from treating PLMs, not as engineering artifacts but as cognitive science and developmental science models. We review assumptions used by researchers to map measures of PLM performance to measures of human performance. We identify potential pitfalls of this approach to understanding human thinking, and we end by enumerating criteria for using PLMs as credible accounts of cognition and cognitive development.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。