用机器人互动视频分析自闭症儿童微表情,提升情绪识别准确率。
A Hybrid Deep Learning Framework for Emotion Recognition in Children with Autism During NAO Robot-Mediated Interaction
- 结合视觉与面部几何特征,用混合深度模型提取情绪线索。
- 在15名自闭症儿童的5万帧数据上实现7类情绪精准分类。
- 首个来自印度的大规模自闭症情绪识别数据集,适合临床应用。
理解自闭症谱系障碍(ASD)儿童在社交互动中的情绪反应仍是发展心理学与人机交互领域的关键挑战。本研究提出一种新型深度学习流水线,用于识别自闭症儿童在人形机器人NAO进行名称攻击事件时的情绪反应,实验在受控环境下进行。数据集包含15名儿童的约5万帧面部视频帧,采用基于ResNet-50的卷积神经网络与三层图卷积网络(GCN)组成的混合模型,融合MediaPipe FaceMesh提取的视觉与几何特征。情绪通过加权集成DeepFace和FER两个模型生成软标签,共七类情绪。最终分类使用基于KL散度优化的融合嵌入。该方法有效捕捉神经多样性儿童的细微情感信号,显著提升对自闭症儿童情感状态的建模能力,为临床及治疗中的人机交互提供有力支持。本工作是印度首个面向自闭症情绪分析的大型真实世界数据集与分析管道,为个性化辅助技术发展奠定基础。
原文摘要 · Abstract (English)
Understanding emotional responses in children with Autism Spectrum Disorder (ASD) during social interaction remains a critical challenge in both developmental psychology and human-robot interaction. This study presents a novel deep learning pipeline for emotion recognition in autistic children in response to a name-calling event by a humanoid robot (NAO), under controlled experimental settings. The dataset comprises of around 50,000 facial frames extracted from video recordings of 15 children with ASD. A hybrid model combining a fine-tuned ResNet-50-based Convolutional Neural Network (CNN) and a three-layer Graph Convolutional Network (GCN) trained on both visual and geometric features extracted from MediaPipe FaceMesh landmarks. Emotions were probabilistically labeled using a weighted ensemble of two models: DeepFace's and FER, each contributing to soft-label generation across seven emotion classes. Final classification leveraged a fused embedding optimized via Kullback-Leibler divergence. The proposed method demonstrates robust performance in modeling subtle affective responses and offers significant promise for affective profiling of ASD children in clinical and therapeutic human-robot interaction contexts, as the pipeline effectively captures micro emotional cues in neurodivergent children, addressing a major gap in autism-specific HRI research. This work represents the first such large-scale, real-world dataset and pipeline from India on autism-focused emotion analysis using social robotics, contributing an essential foundation for future personalized assistive technologies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。