用大模型自动生成认知模型,省去人工设计难题。
Generating Computational Cognitive Models using Large Language Models
- 给定任务和数据,让大模型生成候选认知模型
- 在四个领域中表现优于或媲美经典模型
- 适合认知科学与人工智能交叉研究者
计算认知模型可形式化认知理论,通过拟合行为数据量化认知过程,并在竞争性理论间做出判断。传统上,这些模型需人工精心构建,耗时耗力且依赖专业知识。近年来,大语言模型(LLMs)在上下文模式识别、跨领域知识迁移及生成可执行代码方面展现出强大能力,为解决上述挑战提供了新路径。本文提出一种名为GeCCo的引导式生成管道:基于任务指令、参与者数据和模板函数,由LLM生成候选模型,再利用保留数据评估并迭代优化。我们在决策、学习、规划和记忆四个认知领域,使用三个不同规模、架构的开源大模型进行了测试。在四个真实人类行为数据集上,生成模型的表现持续优于或等同于文献中最佳的领域特定模型。结果表明,大模型能生成概念合理、性能媲美甚至超越现有最优模型的认知模型,适用于多样化的任务场景。
原文摘要 · Abstract (English)
Computational cognitive models, which formalize theories of cognition, enable researchers to quantify cognitive processes and arbitrate between competing theories by fitting models to behavioral data. Traditionally, these models are handcrafted, which requires significant domain knowledge, coding expertise, and time investment. However, recent advances in machine learning offer solutions to these challenges. In particular, Large Language Models (LLMs) have demonstrated remarkable capabilities for in-context pattern recognition, leveraging knowledge from diverse domains to solve complex problems, and generating executable code that can be used to facilitate the generation of cognitive models. Building on this potential, we introduce a pipeline for Guided generation of Computational Cognitive Models (GeCCo). Given task instructions, participant data, and a template function, GeCCo prompts an LLM to propose candidate models, fits proposals to held-out data, and iteratively refines them based on feedback constructed from their predictive performance. We benchmark this approach across four different cognitive domains -- decision making, learning, planning, and memory -- using three open-source LLMs, spanning different model sizes, capacities, and families. On four human behavioral data sets, the LLM generated models that consistently matched or outperformed the best domain-specific models from the cognitive science literature. Taken together, our results suggest that LLMs can generate cognitive models with conceptually plausible theories that rival -- or even surpass -- the best models from the literature across diverse task domains.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。