用新框架提升大模型对教学意图的理解能力,让AI更懂如何教人。
Beyond Skepticism: Evaluating LLMs Pedagogical Intent Reasoning with the Adaptive Pedagogical Vigilance Framework

- 构建贝叶斯推理引擎,让模型反向推断教学内容背后的意图
- 在真实教学语料上表现优异,与人类判断相关性高达0.958
- 适合教育AI、智能辅导系统研发者使用
大语言模型在理解教学沟通中的教学意图方面仍不充分,尤其在翻译教学等教育场景中。为此,我们提出自适应教学警觉(APV)框架,将沟通警觉重构为通过意图推断优化学习的自适应机制。APV通过贝叶斯教学意图推理引擎(PIIE)建模教师如何选择内容以最大化教学效用,并让警觉的学习者逆向推理潜在的教学配置——包括体裁、立场和激励。我们在三层次评估体系上验证:区分教学体裁、推理结构化教学设置、泛化至真实教育话语。实验表明,该框架显著提升模型警觉性,在区分教学与非教学内容上表现最强,与人类判断高度相关(r=0.958),且在自然语料上优于基线方法。本工作建立统一框架,用于评估与增强大模型对教学动机的理解,推动更可靠的AI辅助学习系统发展。
原文摘要 · Abstract (English)
The capacity of Large Language Models (LLMs) to reason about pedagogical intent within instructional communication remains underexplored, particularly in educational domains such as translation pedagogy. To address this, we propose the \textbf{Adaptive Pedagogical Vigilance (APV)} framework, a novel computational formalism that reframes communicative vigilance as an adaptive mechanism for optimizing learning through intent inference. APV formalizes the problem via a Bayesian Pedagogical Intent Inference Engine (PIIE), which models how instructors select content to maximize pedagogical utility and how vigilant learners should inversely reason about latent instructional configurations -- encompassing genre, stance, and incentives. We evaluate APV through a three-tier hierarchy: distinguishing instructional genre, reasoning about structured pedagogical setups, and generalizing to authentic educational discourse. Experiments on leading LLMs (e.g., GPT-4o, Claude 3.5) show that APV substantially improves model vigilance. It achieves the strongest discrimination between pedagogical and exposure-based content, correlates highly with human judgments ($r=0.958$), and maintains robust performance on naturalistic data where baseline methods degrade. This work establishes a unified framework for assessing and enhancing LLMs' understanding of pedagogical motives, advancing the development of more reliable AI-assisted learning systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。