提出AU-TTT模型,用测试时训练提升面部动作单元检测泛化能力。
AU-TTT: Vision Test-Time Training model for Facial Action Unit Detection
- 引入双向测试时训练块,优化图像扫描与区域关注机制。
- 在跨数据集场景下性能优于现有方法,减少过拟合问题。
- 适合需要高泛化性的表情分析应用,如人机交互、心理评估。
面部动作单元(AUs)检测是客观表情分析的核心,在情感计算中至关重要。然而,其面临标注成本高、数据集稀缺等挑战,导致现有方法易过拟合,跨数据集性能显著下降。为解决此问题,本文提出专用于AU检测的视觉主干网络AU-TTT,引入双向测试时训练(TTT)模块,优化图像扫描机制,并设计面向AU的感兴趣区域(RoI)扫描策略以捕捉细微面部特征。该方法在自建和公开数据集上均实现良好表现,尤其在跨域场景下展现更强鲁棒性。实验表明,所提方法在多个基准上达到或超越当前最优水平,有效缓解了泛化瓶颈。
原文摘要 · Abstract (English)
Facial Action Units (AUs) detection is a cornerstone of objective facial expression analysis and a critical focus in affective computing. Despite its importance, AU detection faces significant challenges, such as the high cost of AU annotation and the limited availability of datasets. These constraints often lead to overfitting in existing methods, resulting in substantial performance degradation when applied across diverse datasets. Addressing these issues is essential for improving the reliability and generalizability of AU detection methods. Moreover, many current approaches leverage Transformers for their effectiveness in long-context modeling, but they are hindered by the quadratic complexity of self-attention. Recently, Test-Time Training (TTT) layers have emerged as a promising solution for long-sequence modeling. Additionally, TTT applies self-supervised learning for iterative updates during both training and inference, offering a potential pathway to mitigate the generalization challenges inherent in AU detection tasks. In this paper, we propose a novel vision backbone tailored for AU detection, incorporating bidirectional TTT blocks, named AU-TTT. Our approach introduces TTT Linear to the AU detection task and optimizes image scanning mechanisms for enhanced performance. Additionally, we design an AU-specific Region of Interest (RoI) scanning mechanism to capture fine-grained facial features critical for AU detection. Experimental results demonstrate that our method achieves competitive performance in both within-domain and cross-domain scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。