arXiv:2412.00740cs.CV2024-12被引 12

动态聚合变压器提升人脸关键点检测精度,尤其擅长复杂姿态与遮挡场景。

Precise Facial Landmark Detection by Dynamic Semantic Aggregation Transformer

  • 通过动态语义感知机制分组样本并激活专属特征路径。
  • 在多尺度特征中挖掘同质信息,消除语义歧义,增强表征能力。
  • 适合需要高精度关键点定位的复杂人脸分析任务。

当前深度神经网络在人脸对齐领域占据主导地位,但通常采用预定义结构预测关键点,倾向于学习通用特征,导致性能平庸——在中性表情下表现良好,但在大姿态或遮挡情况下表现不佳。此外,它们难以有效处理不同尺度特征间的语义鸿沟与模糊性,阻碍高效特征学习。为此,本文提出动态语义聚合变压器(DSAT),实现更具区分性和代表性的特征学习。首先设计动态语义感知(DSA)模型,根据特征通道的语义相关性将样本划分为子集,并激活对应路径,从而学习特定于子集的特征。其次提出动态语义专业化(DSS)模型,挖掘多尺度特征中的同质信息,消除语义差距与模糊性,提升表征能力。最后将DSA与DSS以动态架构和动态参数方式融合至DSAT,使模型能学习更专业的特征,实现更高精度的人脸对齐。实验表明,更难样本可通过激活更多特征通道来处理。在多个主流人脸对齐数据集上的大量实验显示,所提方法优于现有最先进模型。代码已开源:https://github.com/GERMINO-LiuHe/DSAT。

原文摘要 · Abstract (English)

At present, deep neural network methods have played a dominant role in face alignment field. However, they generally use predefined network structures to predict landmarks, which tends to learn general features and leads to mediocre performance, e.g., they perform well on neutral samples but struggle with faces exhibiting large poses or occlusions. Moreover, they cannot effectively deal with semantic gaps and ambiguities among features at different scales, which may hinder them from learning efficient features. To address the above issues, in this paper, we propose a Dynamic Semantic-Aggregation Transformer (DSAT) for more discriminative and representative feature (i.e., specialized feature) learning. Specifically, a Dynamic Semantic-Aware (DSA) model is first proposed to partition samples into subsets and activate the specific pathways for them by estimating the semantic correlations of feature channels, making it possible to learn specialized features from each subset. Then, a novel Dynamic Semantic Specialization (DSS) model is designed to mine the homogeneous information from features at different scales for eliminating the semantic gap and ambiguities and enhancing the representation ability. Finally, by integrating the DSA model and DSS model into our proposed DSAT in both dynamic architecture and dynamic parameter manners, more specialized features can be learned for achieving more precise face alignment. It is interesting to show that harder samples can be handled by activating more feature channels. Extensive experiments on popular face alignment datasets demonstrate that our proposed DSAT outperforms state-of-the-art models in the literature.Our code is available at https://github.com/GERMINO-LiuHe/DSAT.

人脸对齐注意力机制特征聚合动态网络

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。