用级联自编码器融合全局与局部动态表情特征,提升识别鲁棒性。
MTCAE-DFER: Multi-Task Cascaded Autoencoder for Dynamic Facial Expression Recognition
- 构建基于ViT的级联解码模块,实现多任务间特征交互。
- 在4个公开数据集上超越现有方法,显著提升动态表情识别性能。
- 适合关注视频表情分析与多任务协同建模的研究者。
本文提出用于动态面部表情识别的多任务级联自编码器(MTCAE-DFER),扩展了基于自编码器的多任务学习框架。该模型设计了一个即插即用的级联解码模块,基于视觉变压器(ViT)架构,利用Transformer解码器机制重构多头注意力模块。前一任务的解码输出作为查询(Q),表征局部动态特征;共享编码器的视频掩码自编码器(VideoMAE)输出同时作为键(K)和值(V),表征全局动态特征。该设计促进相关任务间全局与局部动态特征的交互。此外,通过自编码器式多任务级联学习,探索动态人脸检测与关键点对表情识别的影响,增强模型泛化能力。在多个公开数据集上的大量消融实验和与先进方法的对比验证了模型的鲁棒性及跨任务全局-局部特征交互的有效性。
原文摘要 · Abstract (English)
This paper expands the cascaded network branch of the autoencoder-based multi-task learning (MTL) framework for dynamic facial expression recognition, namely Multi-Task Cascaded Autoencoder for Dynamic Facial Expression Recognition (MTCAE-DFER). MTCAE-DFER builds a plug-and-play cascaded decoder module, which is based on the Vision Transformer (ViT) architecture and employs the decoder concept of Transformer to reconstruct the multi-head attention module. The decoder output from the previous task serves as the query (Q), representing local dynamic features, while the Video Masked Autoencoder (VideoMAE) shared encoder output acts as both the key (K) and value (V), representing global dynamic features. This setup facilitates interaction between global and local dynamic features across related tasks. Additionally, this proposal aims to alleviate overfitting of complex large model. We utilize autoencoder-based multi-task cascaded learning approach to explore the impact of dynamic face detection and dynamic face landmark on dynamic facial expression recognition, which enhances the model's generalization ability. After we conduct extensive ablation experiments and comparison with state-of-the-art (SOTA) methods on various public datasets for dynamic facial expression recognition, the robustness of the MTCAE-DFER model and the effectiveness of global-local dynamic feature interaction among related tasks have been proven.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。