用对抗式多智能体机制提升AI导师的教育可靠性。
Hierarchical Pedagogical Oversight: A Multi-Agent Adversarial Framework for Reliable AI Tutoring
- 分层设计多个专家代理，先提炼对话上下文，再进行五幕制教育辩论。
- 在1214条中学数学对话上，80亿参数模型F1达0.845，优于GPT-4o。
- 适合资源受限场景下部署可信赖的低算力教育助手。
大语言模型正被用于缓解师资短缺，但常缺乏教学推理能力，易认可错误答案或直接给答案阻碍学习。本文提出分层教学监督（HPO）框架，将结构化对抗合成引入教育评估。不同于趋于表面共识的协作式多智能体系统，HPO通过角色分离实现辩证式分工：专业代理先提炼对话上下文，再由对立的教学批评者展开五幕制辩论。在包含1,214条中学数学对话的MRBench数据集上，我们的8B参数模型达到宏平均F1为0.845，较GPT-4o的0.812高出3.3%，且仅需其20分之一的参数量。结果表明，对抗推理是实现低算力、高可靠性教育监管的关键机制。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are increasingly deployed as automated tutors to address educator shortages; however, they often fail at pedagogical reasoning, frequently validating incorrect student solutions (sycophancy) or providing overly direct answers that hinder learning. We introduce Hierarchical Pedagogical Oversight (HPO), a framework that adapts structured adversarial synthesis to educational assessment. Unlike cooperative multi-agent systems that often drift toward superficial consensus, HPO enforces a dialectical separation of concerns: specialist agents first distill dialogue context, which then grounds a moderated, five-act debate between opposing pedagogical critics. We evaluate this framework on the MRBench dataset of 1,214 middle-school mathematics dialogues. Our 8B-parameter model achieves a Macro F1 of 0.845, outperforming GPT-4o (0.812) by 3.3% while using 20 times fewer parameters. These results establish adversarial reasoning as a critical mechanism for deploying reliable, low-compute pedagogical oversight in resource-constrained environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。