用视频特征分析德语手语童话的情感，发现面部与肢体动作同等重要。
Sentiment Analysis of German Sign Language Fairy Tales
- 通过多模型投票标注文本情感，达成0.781的标注一致性。
- 从手语视频提取面部与身体动作特征，用XGBoost模型预测情感。
- 发现眉毛、嘴部及臀部、肘部、肩部运动均影响情感判断。
我们构建了一个德语手语（DGS）童话的情感分析数据集和模型。首先，利用四个大语言模型对德语童话文本片段进行三类情感（负面、中性、正面）分析，并通过多数投票获得0.781的克里彭多夫α值，表明标注一致性良好。其次，使用MediaPipe从对应的DGS视频片段中提取面部与身体运动特征。最后，训练一个可解释的模型（基于XGBoost），根据视频特征预测情感，平均平衡准确率达0.631。对关键特征的深入分析显示，除眉毛和嘴部动作外，臀部、肘部和肩部的运动在情感判别中也起重要作用，表明手语中面部与身体动作在情感传达中具有同等意义。
原文摘要 · Abstract (English)
We present a dataset and a model for sentiment analysis of German sign language (DGS) fairy tales. First, we perform sentiment analysis for three levels of valence (negative, neutral, positive) on German fairy tales text segments using four large language models (LLMs) and majority voting, reaching an inter-annotator agreement of 0.781 Krippendorff's alpha. Second, we extract face and body motion features from each corresponding DGS video segment using MediaPipe. Finally, we train an explainable model (based on XGBoost) to predict negative, neutral or positive sentiment from video features. Results show an average balanced accuracy of 0.631. A thorough analysis of the most important features reveal that, in addition to eyebrows and mouth motion on the face, also the motion of hips, elbows, and shoulders considerably contribute in the discrimination of the conveyed sentiment, indicating an equal importance of face and body for sentiment communication in sign language.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。