arXiv:2605.16283cs.CYcs.AI2026-05

AI提升任务完成效率,但难测其对能力成长的影响。

Toward Measuring AI's Effects on Skill Formation: The Stock-Formation Gap

  • 区分使用现有技能与形成新能力的测量差异
  • 真实场景中AI集中于高技能任务,但未评估长期能力提升
  • 提出将使用记录与独立测试结合的研究方案

大规模AI部署数据与受控学习实验揭示了同一种技术的不同后果。部署遥测数据显示,AI主要应用于高技能工作,频繁支持即时任务完成;然而,这些数据仅观察任务、交互模式和输出结果,无法判断用户是否真正具备独立完成任务的能力。受控研究虽能更直接测量独立能力,但样本范围有限,且结果高度依赖交互设计。本文将此差异定义为‘存量-形成’测量缺口:现有系统更易观测已有技能的使用,却难以捕捉未来能力的形成。由于能力形成是社会应对技术变革的关键机制,这一缺口影响远超课堂范畴。通过分析证据的识别强度,本文以公开部署数据为例说明该缺口,并识别出连接交互痕迹与无协助保留及迁移能力之间的缺失桥梁。随后提出一项研究计划,将知情使用记录与独立评估相联结,并实验性地改变AI提供答案、提示、反馈或评价的方式。本文主张并非已证明AI会削弱群体技能形成,而是现有测量手段无法确定这一点,而该问题既可测量也可设计。

原文摘要 · Abstract (English)

Large-scale AI deployment data and controlled learning experiments characterize different consequences of the same technology. Deployment telemetry shows that AI use is concentrated in skilled work and frequently supports immediate task performance. It observes tasks, interaction patterns, and outputs, however, not whether users become more capable of performing those tasks independently. Controlled studies measure independent capability more directly, but only in narrower populations and settings, with outcomes that vary substantially by interaction design. We formulate this discrepancy as a stock--formation measurement gap: current systems observe the use of existing expertise more readily than the formation of future expertise. Because formation has historically been society's recovery mechanism through technological change, the gap matters well beyond any single classroom. We synthesize the experimental and observational evidence by identification strength, use public deployment data as a descriptive illustration of the gap, and identify the missing bridge between interaction traces and unassisted retention and transfer. We then propose a research program that links consented usage records to independent assessments while experimentally varying whether AI supplies answers, hints, feedback, or evaluation. The claim is not that AI has been shown to erode skill formation at population scale. It is that existing measurement cannot determine whether it does, and that this question is both measurable and designable.

AI教育技能形成测量方法

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。