用真实工作场景评估普通人用AI的能力,比考编程更有效。
AI Literacy Assessment Revisited: A Task-Oriented Approach Aligned with Real-world Occupations
- 以实际工作任务为基准设计评估工具
- 实战任务比传统测试更能反映真实应用能力
- 适合非技术背景人员的AI素养培训与评估
随着人工智能系统在职场日益普及,亟需为非STEM背景的工作者提供有效且负责任使用AI工具的技能训练,即提升其AI素养。然而,当前主流的AI素养定义和评估多聚焦于编程、数学、统计等基础技术知识,忽视了解释模型输出、选择合适工具、识别伦理问题等实际应用能力,导致评估与真实工作需求脱节。本文提出一种面向工作任务的AI素养评估模型,基于职场中有效使用AI所需的核心能力构建。研究在美军海军机器人训练项目中开发了一套新型评估工具及配套形成性评估,并包含实践任务与情境选择题,模拟真实工作中的AI应用。结果显示,竞赛中的情境任务作为应用型评估指标,优于以往研究采用或自主开发的测试。研究建议,在培训非技术背景人员从事AI融合岗位时,应侧重考察高度情境化的实操技能,而非抽象的技术知识。
原文摘要 · Abstract (English)
As artificial intelligence (AI) systems become ubiquitous in professional contexts, there is an urgent need to equip workers, often with backgrounds outside of STEM, with the skills to use these tools effectively as well as responsibly, that is, to be AI literate. However, prevailing definitions and therefore assessments of AI literacy often emphasize foundational technical knowledge, such as programming, mathematics, and statistics, over practical knowledge such as interpreting model outputs, selecting tools, or identifying ethical concerns. This leaves a noticeable gap in assessing someone's AI literacy for real-world job use. We propose a work-task-oriented assessment model for AI literacy which is grounded in the competencies required for effective use of AI tools in professional settings. We describe the development of a novel AI literacy assessment instrument, and accompanying formative assessments, in the context of a US Navy robotics training program. The program included training in robotics and AI literacy, as well as a competition with practical tasks and a multiple choice scenario task meant to simulate use of AI in a job setting. We found that, as a measure of applied AI literacy, the competition's scenario task outperformed the tests we adopted from past research or developed ourselves. We argue that when training people for AI-related work, educators should consider evaluating them with instruments that emphasize highly contextualized practical skills rather than abstract technical knowledge, especially when preparing workers without technical backgrounds for AI-integrated roles.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。