评估AI助判后人还能独立判断多远,避免依赖工具后退步。
Epistemic Transfer in AI-Assisted Verification: A Framework and Evaluation Protocol
- 提出‘认知迁移’概念,区分工具使用与独立判断能力
- 设计ETE和TRC指标,量化用户离开工具后的表现变化
- 提供可落地的实验协议,适合在线研究与真实场景测试
当前评估AI辅助判断工具时,通常只看工具在场时的表现。本文提出新问题:用过工具后,用户离开工具仍能独立处理新信息的能力如何?这称为‘认知迁移’。本文有三项贡献:第一,将认知迁移与纠正效果、信任度、依赖性及人机协作绩效等区分开;第二,提出两个核心指标——认知迁移效应(ETE),比较不同条件下延迟后独立表现;工具移除成本(TRC),衡量工具突然消失时性能下降程度;第三,构建一套可操作的评估协议,包含答案先行与证据先行的AI条件,以及主动练习与无练习对照组,结合延后测试、行为数据和个体/项目层面分析。通过ETE与TRC构成的诊断空间,可识别出能力提升、工具依赖、认知惰化或临时借用四种状态。关键不在于每个工具都必须教学,而在于当独立判断重要时,应检验工具留下的长期影响。
原文摘要 · Abstract (English)
AI tools that help people judge online claims are usually evaluated while the tool is present. This paper asks a different question: after using such a tool, what can the user still do on their own? I call this epistemic transfer. It refers to the effect of prior AI-assisted verification on later unassisted performance on new claims. In this paper, I make three contributions. First, I distinguish epistemic transfer from nearby outcomes such as correction effects, trust, reliance, and human--AI team performance. Second, I introduce two simple quantities for studying it: the Epistemic Transfer Effect (ETE), which compares delayed unassisted performance across conditions, and Tool-Removal Cost (TRC), which measures the immediate drop in performance when the tool is taken away. Third, I turn these ideas into a practical evaluation protocol that can be used in online experiments or field studies. The protocol combines answer-first and evidence-first AI conditions with active-practice and no-practice controls, delayed tests on held-out claims, behavioral measures, and participant- and item-level analyses. Putting ETE and TRC together yields a diagnostic space that separates capability building, capability plus tool advantage, epistemic inertness or de-skilling, and verification on loan. The point is not that every AI tool must teach. The point is that when independent judgment matters, we should test not only whether a tool helps now, but also what it leaves behind.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。