arXiv:2409.00240cs.CVcs.AI2024-09被引 2

用一张中性表情图校准人脸,提升面部动作单元识别准确率

One-Frame Calibration with Siamese Network in Facial Action Unit Recognition

  • 通过单帧中性表情作为参考,构建孪生网络进行校准
  • 在DISFA等数据集上显著降低皱纹、眉毛位置等外观偏差影响
  • 适合需高精度识别的个性化面部分析场景

自动面部动作单元(AU)识别广泛应用于表情分析。现有系统多追求跨个体非校准泛化(NCG),但在未见人脸情况下,仅凭单张图像难以准确推断AU激活——即使人类专家也常受面部特征差异干扰。为此,我们提出单帧校准(OFC)策略:以每张人脸的中性表情图为参考进行校准。基于此,构建了校准型孪生网络(CSN),并采用简单iResNet-50(IR50)骨干网络。在DISFA、DISFA+和UNBC-McMaster数据集上验证表明,OFC CSN-IR50模型(a)显著提升IR50性能,缓解皱纹、眉位、胡须等外观偏差;(b)优于基线减法的朴素OFC方法及微调版本;(c)在AU强度估计与检测任务中均超越当前最优的NCG模型。代码已开源。

原文摘要 · Abstract (English)

Automatic facial action unit (AU) recognition is used widely in facial expression analysis. Most existing AU recognition systems aim for cross-participant non-calibrated generalization (NCG) to unseen faces without further calibration. However, due to the diversity of facial attributes across different identities, accurately inferring AU activation from single images of an unseen face is sometimes infeasible, even for human experts -- it is crucial to first understand how the face appears in its neutral expression, or significant bias may be incurred. Therefore, we propose to perform one-frame calibration (OFC) in AU recognition: for each face, a single image of its neutral expression is used as the reference image for calibration. With this strategy, we develop a Calibrating Siamese Network (CSN) for AU recognition and demonstrate its remarkable effectiveness with a simple iResNet-50 (IR50) backbone. On the DISFA, DISFA+, and UNBC-McMaster datasets, we show that our OFC CSN-IR50 model (a) substantially improves the performance of IR50 by mitigating facial attribute biases (including biases due to wrinkles, eyebrow positions, facial hair, etc.), (b) substantially outperforms the naive OFC method of baseline subtraction as well as (c) a fine-tuned version of this naive OFC method, and (d) also outperforms state-of-the-art NCG models for both AU intensity estimation and AU detection. The code is available at https://github.com/ShuangquanFeng/CSN.

面部识别动作单元孪生网络校准

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。