教师自评与实际AI能力常不匹配,研究提出双评估工具。
How to Assess AI Literacy: Misalignment Between Self-Reported and Objective-Based Measures
- 构建自评与客观测试两类量表,基于概念、应用、评估、伦理四维度。
- 六类教师画像中,超评者占比近半,无经验者普遍低自评低实测。
- 工具可诊断教师需求,支持个性化培训与规模化学习分析。
AI在中小学教育中的普及凸显了对教师AI素养心理测量工具的需求。现有研究多采用自评(SR)或客观测评(OB)单一方式,缺乏在统一框架下对比感知与实际能力的研究,限制了学习分析的可扩展性与基于学习者画像的教学设计发展。本研究在概念、使用、评估、伦理四维框架下开发并验证了SR与OB量表。验证性因子分析显示良好的构念效度与信度,拟合指数可接受。结果发现SR与OB因素间相关性较低。潜剖面分析识别出六种典型群体:高估型(SR > OB)、低估型(SR < OB)、匹配型(SR ≈ OB),以及一类无经验教师组成的低自评-低实测群体。理论上,该研究扩展了现有AI素养框架,验证了两类量表在共同维度上的有效性;实践中,工具可作为专业发展诊断工具,支持成长追踪、需求画像,并实现面向教师子群体的可扩展学习分析干预。
原文摘要 · Abstract (English)
The widespread adoption of Artificial Intelligence (AI) in K-12 education highlights the need for psychometrically-tested measures of teachers' AI literacy. Existing work has primarily relied on either self-report (SR) or objective-based (OB) assessments, with few studies aligning the two within a shared framework to compare perceived versus demonstrated competencies or examine how prior AI literacy experience shapes this relationship. This gap limits the scalability of learning analytics and the development of learner profile-driven instructional design. In this study, we developed and evaluated SR and OB measures of teacher AI literacy within the established framework of Concept, Use, Evaluate, and Ethics. Confirmatory factor analyses support construct validity with good reliability and acceptable fit. Results reveal a low correlation between SR and OB factors. Latent profile analysis identified six distinct profiles, including overestimation (SR > OB), underestimation (SR < OB), alignment (SR close to OB), and a unique low-SR/low-OB profile among teachers without AI literacy experience. Theoretically, this work extends existing AI literacy frameworks by validating SR and OB measures on shared dimensions. Practically, the instruments function as diagnostic tools for professional development, supporting AI-informed decisions (e.g., growth monitoring, needs profiling) and enabling scalable learning analytics interventions tailored to teacher subgroups.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。