通过捕捉表情变化趋势,提升半监督面部动作单元强度估计的鲁棒性。
Trend-Aware Supervision: On Learning Invariance for Semi-Supervised Facial Action Unit Intensity Estimation
- 利用关键帧中的趋势信息作为额外监督信号
- 在BP4D和DISFA数据集上显著提升估计精度
- 无需额外计算成本,适合实际部署
随着面部行为分析需求增加,仅使用关键帧标注的半监督面部动作单元(AU)强度估计成为缓解标注负担的有效方案。然而,标注缺失使动作单元共现和个体差异导致的虚假相关问题更加突出,造成强度估计不稳健、跨个体偏差大。我们发现关键帧标注中蕴含的趋势信息可作为额外监督,提升训练时对特定动作单元面部变化趋势的感知,是学习不变特征的关键。为此,提出趋势感知监督(TAS),包含三种趋势意识:内部趋势排序意识、内部趋势速度意识和跨个体趋势意识。TAS通过增强趋势感知,学习表征对应面部变化的特定于动作单元的特征,实现强度估计的不变性。在两个常用基准数据集BP4D和DISFA上的实验表明,每种趋势意识均有效。在趋势感知监督下,性能提升且推理阶段无额外计算或存储开销。
原文摘要 · Abstract (English)
With the increasing need for facial behavior analysis, semi-supervised AU intensity estimation using only keyframe annotations has emerged as a practical and effective solution to relieve the burden of annotation. However, the lack of annotations makes the spurious correlation problem caused by AU co-occurrences and subject variation much more prominent, leading to non-robust intensity estimation that is entangled among AUs and biased among subjects. We observe that trend information inherent in keyframe annotations could act as extra supervision and raising the awareness of AU-specific facial appearance changing trends during training is the key to learning invariant AU-specific features. To this end, we propose \textbf{T}rend-\textbf{A}ware \textbf{S}upervision (TAS), which pursues three kinds of trend awareness, including intra-trend ranking awareness, intra-trend speed awareness, and inter-trend subject awareness. TAS alleviates the spurious correlation problem by raising trend awareness during training to learn AU-specific features that represent the corresponding facial appearance changes, to achieve intensity estimation invariance. Experiments conducted on two commonly used AU benchmark datasets, BP4D and DISFA, show the effectiveness of each kind of awareness. And under trend-aware supervision, the performance can be improved without extra computational or storage costs during inference.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。