arXiv:2508.03609cs.CV2025-08被引 5

用事件相机提升表情识别,迁移学习让模型更准更快。

evTransFER: A Transfer Learning Framework for Event-based Facial Expression Recognition

  • 用对抗生成训练面部动态特征提取器,迁移权重到识别任务。
  • e-CK+数据集达93.6%准确率,NEFER上达76.7%,优于从零训练。
  • 提出TIE事件表示和LSTM结构,适合低延迟、高动态场景应用。

事件相机是仿生传感器,能以微秒级延迟、高时间分辨率和高动态范围异步捕捉像素强度变化,提供场景的时空动态信息。本文提出evTransFER,一种基于迁移学习的事件相机面部表情识别框架。核心贡献是一个用于编码面部时空动态的特征提取器,通过在面部重建任务上训练对抗生成模型,并将编码器权重迁移到表情识别系统中。我们证明该迁移学习方法显著优于从头训练网络。还设计了一种结合LSTM的架构以捕捉长期表情动态,并引入一种新型事件表示TIE。在合成数据集e-CK+和真实类脑数据集NEFER上进行评估。在e-CK+上达到93.6%识别率,超越现有最佳方法;在含真实传感器噪声和稀疏活动的NEFER上,准确率达76.7%。两种情况下结果均优于当前方法,也高于从零训练模型。

原文摘要 · Abstract (English)

Event-based cameras are bio-inspired sensors that asynchronously capture pixel intensity changes with microsecond latency, high temporal resolution, and high dynamic range, providing information on the spatiotemporal dynamics of a scene. We propose evTransFER, a transfer learning-based framework for facial expression recognition using event-based cameras. The main contribution is a feature extractor designed to encode facial spatiotemporal dynamics, built by training an adversarial generative method on facial reconstruction and transferring the encoder weights to the facial expression recognition system. We demonstrate that the proposed transfer learning method improves facial expression recognition compared to training a network from scratch. We propose an architecture that incorporates an LSTM to capture longer-term facial expression dynamics and introduces a new event-based representation called TIE. We evaluated the framework using both the synthetic event-based facial expression database e-CK+ and the real neuromorphic dataset NEFER. On e-CK+, evTransFER achieved a recognition rate of 93.6\%, surpassing state-of-the-art methods. For NEFER, which comprises event sequence with real sensor noise and sparse activity, the proposed transfer learning strategy achieved an accuracy of up to 76.7\%. In both datasets, the outcomes surpassed current methodologies and exceeded results when compared with models trained from scratch.

事件相机表情识别迁移学习类脑计算

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。