让数学助教读懂学生情绪，自动调整教学策略。
MathBuddy: A Multimodal System for Affective Math Tutoring
- 融合对话文本与面部表情，实时捕捉学生情绪。
- 在8个教学维度上提升23点胜率，整体评分提高3点。
- 适合教育AI、情感计算与个性化学习研究者。
大语言模型驱动的对话系统正快速改变教育科技格局。然而，现有顶尖学习模型普遍忽略学生的心理状态。教育心理学研究表明，积极或消极情绪会影响学习能力。为此，我们提出MathBuddy——一个具备情绪感知能力的LLM数学助教，能动态建模学生情绪，并据此匹配相应教学策略，使师生对话更具同理心。学生情绪通过对话内容与面部表情双重采集，多模态信息融合后触发情感感知响应。我们在八项教学维度上进行自动评估，并开展用户研究。结果显示，胜率提升23点，整体DAMR评分提升3点，有力支持了通过建模情绪可增强LLM助教教学能力的假设。数据集与代码已公开：https://github.com/ITU-NLP/MathBuddy。
原文摘要 · Abstract (English)
The rapid adoption of LLM-based conversational systems is already transforming the landscape of educational technology. However, the current state-of-the-art learning models do not take into account the student's affective states. Multiple studies in educational psychology support the claim that positive or negative emotional states can impact a student's learning capabilities. To bridge this gap, we present MathBuddy, an emotionally aware LLM-powered Math Tutor, which dynamically models the student's emotions and maps them to relevant pedagogical strategies, making the tutor-student conversation a more empathetic one. The student's emotions are captured from the conversational text as well as from their facial expressions. The student's emotions are aggregated from both modalities to confidently prompt our LLM Tutor for an emotionally-aware response. We have evaluated our model using automatic evaluation metrics across eight pedagogical dimensions and user studies. We report a massive 23 point performance gain using the win rate and a 3 point gain at an overall level using DAMR scores which strongly supports our hypothesis of improving LLM-based tutor's pedagogical abilities by modeling students' emotions. Our dataset and code are available at: https://github.com/ITU-NLP/MathBuddy .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。