针对西语老年群体的视频访谈,评估情感计算模型表现并构建新数据集。
Experimenting with Affective Computing Models in Video Interviews with Spanish-speaking Older Adults
- 在真实访谈视频中测试面部表情、文本情绪与微笑检测模型。
- 发现人工标注与模型预测一致性差,跨模态结果不一致,个体差异大。
- 适合做适老化虚拟助手开发的研究者参考。
理解老年人的情绪信号对设计支持其福祉的虚拟助手至关重要。然而现有情感计算模型存在两大局限:(1)缺乏代表老年人的数据集,尤其是非英语人群;(2)在年轻或同质化人群中训练的模型泛化能力差。为此,本研究评估了当前主流情感计算模型——包括面部表情识别、文本情绪分析和微笑检测——在老年人与真人或虚拟角色互动的视频中的表现。作为该研究的一部分,我们引入了一个新数据集,包含西班牙语老年群体在人机视频访谈中的数据。通过三项综合分析,我们考察了(1)人工标注与自动模型输出的一致性,(2)不同模态间模型输出的关系,以及(3)个体情绪信号的差异。基于巫师奥兹(WoZ)数据集和新收集的数据集,我们发现人工标注与模型预测之间一致性有限,各模态间结果一致性弱,个体间差异显著。这些结果揭示了通用情绪感知模型的不足,强调未来系统需纳入个人差异与文化特征。
原文摘要 · Abstract (English)
Understanding emotional signals in older adults is crucial for designing virtual assistants that support their well-being. However, existing affective computing models often face significant limitations: (1) limited availability of datasets representing older adults, especially in non-English-speaking populations, and (2) poor generalization of models trained on younger or homogeneous demographics. To address these gaps, this study evaluates state-of-the-art affective computing models -- including facial expression recognition, text sentiment analysis, and smile detection -- using videos of older adults interacting with either a person or a virtual avatar. As part of this effort, we introduce a novel dataset featuring Spanish-speaking older adults engaged in human-to-human video interviews. Through three comprehensive analyses, we investigate (1) the alignment between human-annotated labels and automatic model outputs, (2) the relationships between model outputs across different modalities, and (3) individual variations in emotional signals. Using both the Wizard of Oz (WoZ) dataset and our newly collected dataset, we uncover limited agreement between human annotations and model predictions, weak consistency across modalities, and significant variability among individuals. These findings highlight the shortcomings of generalized emotion perception models and emphasize the need of incorporating personal variability and cultural nuances into future systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。