Centaur虽能预测人类行为,但生成行为仍与真人有差异。
Not Yet AlphaFold for the Mind: Evaluating Centaur as a Synthetic Participant
- 用160项实验数据微调大模型,模拟人类认知行为
- 预测准确率高,但生成反应模式与真实人类显著不同
- 适合研究自动化认知科学,但尚不满足可靠模拟标准
模拟器已彻底改变自然科学的研究方式。通过生成可靠逼近现实现象的数据,它们加速了假设检验和实验设计优化。这在化学领域体现得最为明显:AlphaFold 能从氨基酸序列预测蛋白质结构,推动分子相互作用、药物靶点和蛋白功能的快速原型开发。在行为科学中,一个可靠的参与者模拟器——即能在认知任务中生成类人行为的系统——将带来类似变革。最近,Binz 等人提出 Centaur,一个在160项实验的人类数据上微调的大语言模型,主张其不仅可作为认知模型,还可用于“虚拟实验原型”,推动自动化认知科学研究。本文回顾参与者模拟器的核心标准,并评估 Centaur 的表现。尽管其预测准确度高,但其生成行为(作为模拟器的关键标准)系统性偏离真实人类数据。这表明,尽管 Centaur 是迈向预测人类行为的重要一步,但它尚未达到可靠参与者模拟器或准确认知模型的标准。
原文摘要 · Abstract (English)
Simulators have revolutionized scientific practice across the natural sciences. By generating data that reliably approximate real-world phenomena, they enable scientists to accelerate hypothesis testing and optimize experimental designs. This is perhaps best illustrated by AlphaFold, a Nobel-prize winning simulator in chemistry that predicts protein structures from amino acid sequences, enabling rapid prototyping of molecular interactions, drug targets, and protein functions. In the behavioral sciences, a reliable participant simulator - a system capable of producing human-like behavior across cognitive tasks - would represent a similarly transformative advance. Recently, Binz et al. introduced Centaur, a large language model (LLM) fine-tuned on human data from 160 experiments, proposing its use not only as a model of cognition but also as a participant simulator for "in silico prototyping of experimental studies", e.g., to advance automated cognitive science. Here, we review the core criteria for a participant simulator and assess how well Centaur meets them. Although Centaur demonstrates strong predictive accuracy, its generative behavior - a critical criterion for a participant simulator - systematically diverges from human data. This suggests that, while Centaur is a significant step toward predicting human behavior, it does not yet meet the standards of a reliable participant simulator or an accurate model of cognition.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。