基于多模态反馈的可解释智能导师系统，助力远程演讲训练
An Interpretable Closed-Loop Intelligent Tutoring System for Multimodal Affective Feedback in Asynchronous Presentation Training
- 构建三层次可解释反馈架构，连接评分、感知与对话辅导
- 在10360段MOOC视频上实现与专家评分相当的准确率（R²=0.48-0.61）
- 204名学习者30天练习后7个维度显著提升，频率越高效果越佳
本文提出一种可解释的闭环智能导师系统（ITS），支持大规模异步演讲训练中的反馈驱动练习。系统基于七维行为锚定评分量表（BARS），采用三层可解释反馈架构，将评分对齐的多模态打分、观众感知的情感诊断与检索增强的对话式辅导相连接，促进刻意练习。系统以XGBoost为骨干，将面部、语音、文本和眼动等多模态输入映射为可追溯至可观测行为线索的证据型反馈。在10,360段MOOC视频上训练，其评分结果与专家评级表现相当（R² = 0.48–0.61，Spearman's rho = 0.69–0.78，MAE = 0.43–0.57）。在204名成人学习者参与的前后测研究中，30天练习期内所有七个BARS维度均显著提升（Cohen's d = 0.39–0.90），控制基线与人口统计因素后，练习频率与最终表现呈强正相关。结果表明，多模态分析输出可通过集成反馈架构系统性转化为可观察的行为改变，推动面向表现能力的可解释且教学有据的智能导师系统发展。
原文摘要 · Abstract (English)
This paper presents an interpretable closed-loop Intelligent Tutoring System (ITS) that supports feedback-guided practice for developing on-camera oral presentation skills at scale. The system operationalizes a seven-dimensional Behaviorally Anchored Rating Scale (BARS) and implements a three-layer interpretable feedback architecture that connects rubric-aligned multimodal scoring, audience-perceived expressive diagnostics, and retrieval-augmented conversational coaching to support deliberate practice. Built on an XGBoost backbone, the ITS maps multimodal inputs (facial, vocal, textual, and oculomotor features) into evidence-based feedback that can be traced back to observable performance cues. Trained on 10,360 Massive Open Online Course (MOOC) video segments, the system achieved rubric-aligned scoring with performance levels comparable to expert ratings (R2 = 0.48-0.61, Spearman's rho = 0.69-0.78, MAE = 0.43-0.57). In a pre-post validation study with 204 adult learners over a 30-day practice window, participants demonstrated significant improvements across all seven BARS dimensions (Cohen's d = 0.39-0.90), with practice frequency showing a strong positive association with posttest performance after controlling for baseline scores and demographics. The results demonstrate how multimodal analytic outputs can be systematically transformed into observable behavioral change through an integrated feedback architecture, advancing explainable and pedagogically grounded ITS design for performance-based competencies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。