arXiv:2409.08027cs.CYcs.HC2024-09中稿 · AAAI

用社会学解释理论让AI给学生反馈更易懂、可行动。

iLLuMinaTE: An LLM-XAI Framework Leveraging Social Science Explanation Theories Towards Actionable Student Performance Feedback

  • 基于认知与社会学理论构建零样本提示链,生成有逻辑的反馈。
  • 89.52%学生更偏好iLLuMinaTE的解释,实测提升理解与行动意愿。
  • 适合教育AI、人机交互领域,尤其关注非技术用户需求。

近年来教育领域的可解释AI研究面临关键挑战:如何让非技术用户(如教师和学生)理解先进AI模型的解释。为此,我们提出iLLuMinaTE,一种受米勒认知解释模型启发的零样本链式提示大模型-可解释AI框架。该框架通过因果关联、解释选择与呈现三个阶段,融合八种社会科学解释理论(如异常条件、佩尔的解释模型、必要性与鲁棒性选择、对比解释等),为在线课程学生生成理论驱动且可操作的反馈。我们对三种大模型(GPT-4o、Gemma2-9B、Llama3-70B)和三种底层XAI方法(LIME、反事实、MC-LIME)生成的21,915条自然语言解释进行了广泛评估,涵盖三门不同在线课程的学生数据。评估包含解释与理论的一致性、可理解性分析,以及114名大学生参与的真实用户偏好研究,含新颖的可行动性模拟。结果表明,学生在89.52%情况下更偏好iLLuMinaTE的解释。本工作提供了一个可直接使用的教育可解释框架,具备向其他以人为本领域的强泛化潜力。

原文摘要 · Abstract (English)

Recent advances in eXplainable AI (XAI) for education have highlighted a critical challenge: ensuring that explanations for state-of-the-art AI models are understandable for non-technical users such as educators and students. In response, we introduce iLLuMinaTE, a zero-shot, chain-of-prompts LLM-XAI pipeline inspired by Miller's cognitive model of explanation. iLLuMinaTE is designed to deliver theory-driven, actionable feedback to students in online courses. iLLuMinaTE navigates three main stages - causal connection, explanation selection, and explanation presentation - with variations drawing from eight social science theories (e.g. Abnormal Conditions, Pearl's Model of Explanation, Necessity and Robustness Selection, Contrastive Explanation). We extensively evaluate 21,915 natural language explanations of iLLuMinaTE extracted from three LLMs (GPT-4o, Gemma2-9B, Llama3-70B), with three different underlying XAI methods (LIME, Counterfactuals, MC-LIME), across students from three diverse online courses. Our evaluation involves analyses of explanation alignment to the social science theory, understandability of the explanation, and a real-world user preference study with 114 university students containing a novel actionability simulation. We find that students prefer iLLuMinaTE explanations over traditional explainers 89.52% of the time. Our work provides a robust, ready-to-use framework for effectively communicating hybrid XAI-driven insights in education, with significant generalization potential for other human-centric fields.

教育AI可解释AI大模型学生反馈

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。