arXiv:2603.18758cs.HCcs.CV2026-03中稿 · publication in IEE…

仅凭演讲者表情与声音,就能预测观众情绪投入和声音魅力。

Dual-Model Prediction of Affective Engagement and Vocal Attractiveness from Speaker Expressiveness in Video Learning

  • 用演讲者面部、眼神、语调和语义特征预测观众情绪投入。
  • 在独立测试集上,情绪投入预测R²达0.85,声音魅力预测R²达0.88。
  • 无需观众数据,适合用于隐私保护的在线教学优化。

本文提出一种以演讲者为中心的情感智能方法,仅依靠演讲者自身的情感表达,即可预测异步视频学习中观众的情绪投入度与声音吸引力。该方法基于大规模开放在线课程(MOOCs)构建的海量数据集,采用两个独立回归模型:一个融合面部动态、眼动特征、语调和认知语义信息以预测情绪投入;另一个仅使用演讲者声学特征预测声音魅力。在跨演讲者测试集上,情绪投入预测的R²达到0.85,声音吸引力预测的R²达0.88,证明演讲者侧多模态特征可有效代表观众整体反馈。研究验证了仅通过演讲者表现即可前瞻性预测观众反应,为可扩展、隐私友好的情感计算应用提供了实证支持。

原文摘要 · Abstract (English)

This paper outlines a machine learning-enabled speaker-centric Emotion AI approach capable of predicting audience-affective engagement and vocal attractiveness in asynchronous video-based learning, relying solely on speaker-side affective expressions. Inspired by the demand for scalable, privacy-preserving affective computing applications, this speaker-centric Emotion AI approach incorporates two distinct regression models that leverage a massive corpus developed within Massive Open Online Courses (MOOCs) to enable affectively engaging experiences. The regression model predicting affective engagement is developed by assimilating emotional expressions emanating from facial dynamics, oculomotor features, prosody, and cognitive semantics, while incorporating a second regression model to predict vocal attractiveness based exclusively on speaker-side acoustic features. Notably, on speaker-independent test sets, both regression models yielded impressive predictive performance (R2 = 0.85 for affective engagement and R2 = 0.88 for vocal attractiveness), confirming that speaker-side affect can functionally represent aggregated audience feedback. This paper provides a speaker-centric Emotion AI approach substantiated by an empirical study discovering that speaker-side multimodal features, including acoustics, can prospectively forecast audience feedback without necessarily employing audience-side input information.

情感计算语音魅力多模态分析在线教育

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。