arXiv:2506.13384cs.AIcs.CY2025-06综述被引 7

用大模型模拟学习心理问卷,验证其在教育研究中的可行性

Delving Into the Psychology of Machines: Exploring the Structure of Self-Regulated Learning via LLM-Generated Survey Responses

  • 用5个大模型生成学习动机问卷回复,分析其心理结构一致性
  • Gemini 2 Flash表现最佳,与理论预期和实证结果高度吻合
  • 适合教育心理学研究者探索新干预方案或补充小样本数据

大型语言模型(LLMs)具备模拟人类响应的潜力,为心理学研究带来新机遇。在自我调节学习(SRL)领域,若能规模化、快速生成可靠的问卷回答,可用于测试干预方案、优化理论模型、扩充稀疏数据集或代表难触及群体。然而,现有研究对LLM生成问卷数据的有效性仍存疑,且聚焦SRL的研究较少,其他领域的结果亦不一致。本研究针对44项动机学习策略问卷(MSLQ;Pintrich & De Groot, 1990),使用GPT-4o、Claude 3.7 Sonnet、Gemini 2 Flash、LLaMA 3.1-8B和Mistral Large五款模型,分析项目分布、理论维度的心理网络结构及潜因子结构的效度。结果显示,Gemini 2 Flash表现最优,具有合理采样变异性,并揭示出与既有理论和实证发现一致的潜在维度与关系。同时,也观察到若干差异与局限,凸显大模型在模拟心理调查数据方面的潜力与当前限制。

原文摘要 · Abstract (English)

Large language models (LLMs) offer the potential to simulate human-like responses and behaviors, creating new opportunities for psychological science. In the context of self-regulated learning (SRL), if LLMs can reliably simulate survey responses at scale and speed, they could be used to test intervention scenarios, refine theoretical models, augment sparse datasets, and represent hard-to-reach populations. However, the validity of LLM-generated survey responses remains uncertain, with limited research focused on SRL and existing studies beyond SRL yielding mixed results. Therefore, in this study, we examined LLM-generated responses to the 44-item Motivated Strategies for Learning Questionnaire (MSLQ; Pintrich \& De Groot, 1990), a widely used instrument assessing students' learning strategies and academic motivation. Particularly, we used the LLMs GPT-4o, Claude 3.7 Sonnet, Gemini 2 Flash, LLaMA 3.1-8B, and Mistral Large. We analyzed item distributions, the psychological network of the theoretical SRL dimensions, and psychometric validity based on the latent factor structure. Our results suggest that Gemini 2 Flash was the most promising LLM, showing considerable sampling variability and producing underlying dimensions and theoretical relationships that align with prior theory and empirical findings. At the same time, we observed discrepancies and limitations, underscoring both the potential and current constraints of using LLMs for simulating psychological survey data and applying it in educational contexts.

心理模拟大模型教育研究问卷生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。