让大模型根据学生认知状态动态调整教学策略,提升理解深度。
UCO: A Multi-Turn Interactive Reinforcement Learning Method for Adaptive Teaching with Large Language Models
- 通过多轮交互强化学习,实时感知学生认知进展。
- 在两个数学评测集上优于11个基线模型,接近闭源先进水平。
- 创新设计双奖励机制,精准引导学生进入最佳学习区间。
大型语言模型正从问答工具转向智能导师,但现有监督微调方法仅学习表层教学模式,缺乏动态适应能力。近期强化学习方法虽有所改进,但仍面临两大挑战:一是仅以答案正确性评估教学效果,无法区分学生是真正理解还是机械复述;二是无法通过对话实时感知学生认知状态变化,难以动态调整教学策略。本文提出单向认知优化(UCO)方法,采用多轮交互式强化学习范式,核心在于两个协同的奖励函数:进展奖励(Progress Reward)捕捉学生从困惑到理解的认知进步,支架奖励(Scaffold Reward)动态识别学生的最近发展区(ZPD),引导教师在该区间内开展有效教学。我们在BigMath和MathTutorBench两个基准上对比了11个基线模型,实验表明UCO模型在同等规模下全面超越基线,性能媲美先进的闭源模型。代码与数据已公开于https://github.com/Mind-Lab-ECNU/UCO。
原文摘要 · Abstract (English)
Large language models (LLMs) are shifting from answer providers to intelligent tutors in educational settings, yet current supervised fine-tuning methods only learn surface teaching patterns without dynamic adaptation capabilities. Recent reinforcement learning approaches address this limitation but face two critical challenges. First, they evaluate teaching effectiveness solely based on whether students produce correct outputs, unable to distinguish whether students genuinely understand or echo teacher-provided answers during interaction. Second, they cannot perceive students' evolving cognitive states in real time through interactive dialogue, thus failing to adapt teaching strategies to match students' cognitive levels dynamically. We propose the Unidirectional Cognitive Optimization (UCO) method to address these challenges. UCO uses a multi-turn interactive reinforcement learning paradigm where the innovation lies in two synergistic reward functions: the Progress Reward captures students' cognitive advancement, evaluating whether students truly transition from confusion to comprehension, while the Scaffold Reward dynamically identifies each student's Zone of Proximal Development (ZPD), encouraging teachers to maintain productive teaching within this zone. We evaluate UCO by comparing it against 11 baseline models on BigMath and MathTutorBench benchmarks. Experimental results demonstrate that our UCO model outperforms all models of equivalent scale and achieves performance comparable to advanced closed-source models. The code and data are available at https://github.com/Mind-Lab-ECNU/UCO.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。