构建教育场景下AI教学解释的风险评估数据集,助力安全可控的AI助教。
AIriskEval-edu: New Dataset for Risk Assessment in AI-mediated K-12 Educational Explanations

- 设计覆盖5类风险维度的标注体系,支持多角色教师风格模拟。
- 包含1639条真实与生成解释,785条带风险定位和描述的可解释标注。
- 验证轻量本地模型可媲美大厂闭源模型,兼顾隐私与效果。
本文提出AIriskEval-edu-db2数据集,用于训练与评估基于大语言模型的审计系统,实现面向K-12教学内容的可解释性教育风险评估。数据集包含170个精选ScienceQA问题的1,639条解释,涵盖科学、语文和社科领域。每道题均配有真人教师解释及11种由LLM模拟的不同教学风险风格的生成解释。研究构建了符合教育标准的五维风险评估框架:事实准确性、深度完整性、聚焦相关性、学段适切性与意识形态偏见。关键贡献在于新增785条结构化可解释性标注,包含风险定位与描述,通过半自动流程结合专家教师验证完成。最后,通过实验对比主流闭源模型与轻量级本地Llama 3.1 8B模型在风险检测与可解释性评估中的表现,验证了在监督微调后本地模型能否逼近甚至超越前沿模型,同时保障教育审计任务中的数据隐私。
原文摘要 · Abstract (English)
This work introduces AIriskEval-edu-db2, a new dataset designed to train and evaluate auditors based on LLMs for an explainable pedagogical risk assessment in instructional content for grades K-12. The dataset comprises 1,639 explanations from 170 curated ScienceQA questions, covering science, language arts, and social sciences. For each question, the dataset includes an explanation written by a human teacher alongside 11 explanations generated by LLM-simulated teacher profiles associated with distinct pedagogical risks. We propose a comprehensive risk rubric aligned with established educational standards that covers five complementary dimensions: factual precision, depth and completeness, focus and relevance, student-level appropriateness, and ideological bias. A key contribution is the addition of 785 explanations with structured explainability annotations, including risk localization and risk description. The annotations are produced through a semi-automatic process with expert teacher validation. Finally, we present validation experiments comparing state-of-the-art proprietary models with a lightweight local Llama 3.1 8B model in both the pedagogical risk detection and the explainability assessment. These experiments evaluate whether supervised fine-tuning on AIriskEval-edu-db2 enables a locally deployable model to approach or outperform stronger frontier models while preserving privacy in educational auditing and assessment tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。