提出可量化的AI使用批判性思维量表,帮助评估用户如何审慎对待生成式AI输出。
Understanding Critical Thinking in Generative Artificial Intelligence Use: Development, Validation, and Correlates of the Critical Thinking in AI Use Scale
- 开发13项量表,涵盖验证、动机和反思三维度,测量用户对AI输出的审慎态度。
- 实证发现高批判性思维者更频繁使用多种验证策略,且在事实核查任务中判断更准确。
- 量表具有跨性别一致性与时间稳定性,适合研究不同群体的AI使用认知差异。
生成式AI日益融入日常学习与工作,但其流畅性、黑箱特性及幻觉倾向要求使用者必须批判性评估输出而非盲目接受。本研究将AI使用中的批判性思维定义为一种倾向:主动验证AI内容与来源、理解模型运作机制与局限,并反思依赖AI的广泛影响。通过六项研究(总样本N = 1341),我们开发并验证了13题项的‘AI使用批判性思维量表’,确立三因子结构(验证、动机、反思)。研究3-4证实该量表具备高因子载荷、内部一致性、性别不变性,以及聚合效度与区分效度;并与开放性、外向性、积极情绪特质及高频使用AI正相关。研究5验证了测试-重测信度。研究6进一步证明准则效度:高分者更常采用多样验证策略,在新型自然情境下基于GPT的聊天机器人事实核查任务中判断更准确,且表现出更深的责任感反思。本研究厘清了人类如何监督生成式AI输出的机制,提供了经验证的量表与生态化范式,支持理论检验、跨群体与纵向研究。
原文摘要 · Abstract (English)
Generative AI tools are increasingly embedded in everyday work and learning, yet their fluency, opacity, and propensity to hallucinate mean that users must critically evaluate AI outputs rather than accept them at face value. The present research conceptualises critical thinking in AI use as a dispositional tendency to verify the source and content of AI-generated information, to understand how models work and where they fail, and to reflect on the broader implications of relying on AI. Across six studies (N = 1341), we developed and validated the 13-item critical thinking in AI use scale and mapped its nomological network. Study 1 generated and content-validated scale items. Study 2 supported a three-factor structure (Verification, Motivation, and Reflection). Studies 3 and 4 confirmed the higher-order model, demonstrated strong factor loadings, internal consistency, sex invariance, convergent and discriminant evidence for validity, and showed that critical thinking in AI use was positively associated with openness, extraversion, positive trait affect, and frequency of AI use. Study 5 supported the scale's test-retest reliability. Lastly, Study 6 demonstrated criterion evidence of validity for the scale, with higher critical thinking in AI use scores predicting more frequent and diverse verification strategies, greater veracity judgement accuracy in a novel and naturalistic GPT-powered AI chatbot fact-checking task, and deeper reflection about responsible AI. The current work clarifies why and how people exercise oversight over generative AI outputs and provides a validated scale and ecologically grounded paradigm to support theory testing, cross-group, and longitudinal research on critical thinking in AI use.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。