arXiv:2503.16532cs.HCcs.AI2025-03被引 4

融合眼动、性格与时间动态,提升对话场景情绪识别准确率

Modelling Emotions in Face-to-Face Setting: The Interplay of Eye-Tracking, Personality, and Temporal Dynamics

  • 整合眼动数据、人格特质与上下文信息建模情绪
  • 感知愉悦度F1达0.76,加入个性特征后内感情绪识别更准
  • 适合做情感计算与人机交互系统优化的研究者参考

准确的情绪识别对实现自然流畅的人机交互至关重要,但在动态对话场景中仍具挑战。本研究展示通过融合眼动信号(瞳孔大小、注视模式)、人格特质(大五人格)及时间动态,可显著提升对感知与内感情绪的检测。73名参与者观看含语音的CREMA-D数据集视频时,同步采集眼动数据、人格评估与自评情绪状态。神经网络模型融合刺激情绪标签等上下文线索,性能优于当前最优方法。感知愉悦度的宏平均F1得分为0.76;加入人格与刺激信息的模型在内感情绪识别上表现显著提升。结果表明,统一生理、个体与上下文因素有助于应对情绪表达的主观性与复杂性。研究为未来情感计算与人机代理系统设计提供依据,推动真实交互中更具适应性的跨个体情绪智能发展。

原文摘要 · Abstract (English)

Accurate emotion recognition is pivotal for nuanced and engaging human-computer interactions, yet remains difficult to achieve, especially in dynamic, conversation-like settings. In this study, we showcase how integrating eye-tracking data, temporal dynamics, and personality traits can substantially enhance the detection of both perceived and felt emotions. Seventy-three participants viewed short, speech-containing videos from the CREMA-D dataset, while being recorded for eye-tracking signals (pupil size, fixation patterns), Big Five personality assessments, and self-reported emotional states. Our neural network models combined these diverse inputs including stimulus emotion labels for contextual cues and yielded marked performance gains compared to the state-of-the-art. Specifically, perceived valence predictions reached a macro F1-score of 0.76, and models incorporating personality traits and stimulus information demonstrated significant improvements in felt emotion accuracy. These results highlight the benefit of unifying physiological, individual and contextual factors to address the subjectivity and complexity of emotional expression. Beyond validating the role of user-specific data in capturing subtle internal states, our findings inform the design of future affective computing and human-agent systems, paving the way for more adaptive and cross-individual emotional intelligence in real-world interactions.

情绪识别眼动追踪人格特质情感计算

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。