分析AI语言学习反馈的失效模式，揭示看似合理实则有害的解释陷阱。
Ceci n'est pas une explication: Evaluating Explanation Failures as Explainability Pitfalls in Language Learning Systems
- 构建六维度评估框架，检验AI反馈的诊断准确性与指导有效性。
- 发现AI解释常表面合理却暗藏错误，长期使用会误导学习者。
- 适合关注AI教育应用安全性的开发者与研究者阅读。
基于全球数百万学习者使用的AI语言学习工具日益普及，其即时个性化反馈虽便捷，却可能以难以察觉的方式失效，长期使用或强化学习误解并损害学习成效。本文介绍L2-Bench的一部分，该基准涵盖六项关键反馈维度：诊断准确性、语用意识、错误成因识别、优先级排序、改进建议及支持自我调节能力。我们分析了AI系统在这些维度上的典型失败表现，指出这些缺陷构成‘可解释性陷阱’——即生成看似有帮助但实质错误的解释，增加学习障碍、人机交互风险及社会情感危害。文章强调语言学习场景放大了此类风险，并呼吁设计更具针对性的评估框架。研究旨在拓展对可解释性陷阱类型及其情境动态的理解，推动开发者设计更安全、可信且有效的AI解释。
原文摘要 · Abstract (English)
AI-powered language learning tools increasingly provide instant, personalised feedback to millions of learners worldwide. However, this feedback can fail in ways that are difficult for learners--and even teachers--to detect, potentially reinforcing misconceptions and eroding learning outcomes over extended use. We present a portion of L2-Bench, a benchmark for evaluating AI systems in language education that includes (but is not limited to) six critical dimensions of effective feedback: diagnostic accuracy, awareness of appropriacy, causes of error, prioritisation, guidance for improvement, and supporting self-regulation. We analyse how AI systems can fail with respect to these dimensions. These failures, which we argue are conducive to "explainability pitfalls," are AI-generated explanations that appear helpful on the surface but are fundamentally flawed, increasing the risk of attainment, human-AI interaction, and socioaffective harms. We discuss how the specific context of language learning amplifies these risks and outline open questions we believe merit more attention when designing evaluation frameworks specifically. Our analysis aims to expand the community's understanding of both the typology of explainability pitfalls and the contextual dynamics in which they may occur in order to encourage AI developers to better design safe, trustworthy, and effective AI explanations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。