arXiv:2601.15914cs.CVcs.HC2026-01

评测现成模型在虚拟形象情绪识别中的实时表现,发现分类阶段存在延迟瓶颈。

The Latency Wall: Benchmarking Off-the-Shelf Emotion Recognition for Real-Time Virtual Avatars

  • 用轻量级YOLO检测人脸,结合通用视觉模型进行表情分类
  • 纯CPU下分类准确率低于23%或延迟超150毫秒,无法满足实时需求
  • 适合关注自闭症干预的VR治疗系统开发者参考

在虚拟现实与人机交互领域,实时情绪识别有望帮助自闭症谱系障碍患者提升社交能力。该任务需严格控制延迟,运动到光子(MTP)延迟须低于140毫秒以保证感知连续性。然而,多数现成深度学习模型更注重准确率,忽视消费级硬件的时序约束。作为实现可及性VR治疗的第一步,我们基于UIBVFED数据集,对零样本面部表情识别(FER)的SOTA模型进行了基准测试。评估了YOLO(v8、v11、v12)的中型与纳米版本用于人脸检测,以及通用视觉变压器如CLIP、SigLIP和ViT-FER。结果显示,在仅使用CPU推理的情况下,对风格化虚拟角色的人脸检测表现稳健(准确率100%),但分类阶段存在“延迟墙”。YOLOv11n在检测上表现最优(约54毫秒)。而通用视觉变压器如CLIP和SigLIP在实时循环中无法达到可行的准确率(<23%)或速度(>150毫秒)。本研究凸显了开发轻量化、领域专用架构的必要性,以实现在治疗场景中的可及性实时人工智能。

原文摘要 · Abstract (English)

In the realm of Virtual Reality (VR) and Human-Computer Interaction (HCI), real-time emotion recognition shows promise for supporting individuals with Autism Spectrum Disorder (ASD) in improving social skills. This task requires a strict latency-accuracy trade-off, with motion-to-photon (MTP) latency kept below 140 ms to maintain contingency. However, most off-the-shelf Deep Learning models prioritize accuracy over the strict timing constraints of commodity hardware. As a first step toward accessible VR therapy, we benchmark State-of-the-Art (SOTA) models for Zero-Shot Facial Expression Recognition (FER) on virtual characters using the UIBVFED dataset. We evaluate Medium and Nano variants of YOLO (v8, v11, and v12) for face detection, alongside general-purpose Vision Transformers including CLIP, SigLIP, and ViT-FER.Our results on CPU-only inference demonstrate that while face detection on stylized avatars is robust (100% accuracy), a "Latency Wall" exists in the classification stage. The YOLOv11n architecture offers the optimal balance for detection (~54 ms). However, general-purpose Transformers like CLIP and SigLIP fail to achieve viable accuracy (<23%) or speed (>150 ms) for real-time loops. This study highlights the necessity for lightweight, domain-specific architectures to enable accessible, real-time AI in therapeutic settings.

情绪识别实时系统虚拟化身自闭症干预

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。