基于预训练模型与嵌入融合，提升面部情绪与暴力行为识别精度
HSEmotion Team at ABAW-10 Competition: Facial Expression Recognition, Valence-Arousal Estimation, Action Unit Detection and Fine-Grained Violence Classification
- 用预训练EfficientNet提取面部嵌入，高置信度直接输出，否则用MLP分类
- 滑动窗口平滑帧级预测，有效降低噪声干扰
- 在四个任务中均超越基线，适合实时情绪与行为分析应用
本文报告了我们在第10届Affective Behavior Analysis in-the-Wild（ABAW）竞赛中的成果。针对帧级面部情绪理解任务（包括面部表情识别、效价-唤醒度估计、动作单元检测），我们提出一种快速方法：基于预训练的EfficientNet情绪识别模型提取面部嵌入。若模型置信度超过阈值，则直接使用其预测结果；否则将嵌入输入在AffWild2数据集上训练的简单多层感知机。对估计的类别得分采用固定大小滑动窗口进行平滑处理，以抑制帧级预测噪声。对于细粒度暴力检测任务，我们测试了多种预训练架构生成的帧嵌入及其聚合方式用于视频分类。在ABAW挑战赛的四个任务上，实验结果表明，该方法显著优于现有基线模型。
原文摘要 · Abstract (English)
This article presents our results for the 10th Affective Behavior Analysis in-the-Wild (ABAW) competition. For frame-wise facial emotion understanding tasks (frame-wise facial expression recognition, valence-arousal estimation, action unit detection), we propose a fast approach based on facial embedding extraction with pre-trained EfficientNet-based emotion recognition models. If the latter model's confidence exceeds a threshold, its prediction is used. Otherwise, we feed embeddings into a simple multi-layered perceptron trained on the AffWild2 dataset. Estimated class-level scores are smoothed in a sliding window of fixed size to mitigate noise in frame-wise predictions. For the fine-grained violence detection task, we examine several pre-trained architectures for frame embeddings and their aggregation for video classification. Experimental results on four tasks from the ABAW challenge demonstrate that our approach significantly improves validation metrics over existing baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。