arXiv:2510.15338cs.CV2025-10中稿 · TMM October 2025被引 5

统一多数据集人脸关键点检测,通过原型网络提升泛化能力。

Proto-Former: Unified Facial Landmark Detection by Prototype Transformer

  • 用可适应的原型编码器和解码器学习跨数据集的关键点特征。
  • 在多个基准数据集上超越现有最佳方法,精度显著提升。
  • 适合需要跨数据集通用的人脸分析任务使用。

深度学习的进展显著提升了人脸关键点检测性能。然而,现有数据集定义的关键点数量各不相同,主流方法通常只能在单一数据集上训练,限制了模型在不同数据集间的泛化能力,阻碍了统一模型的发展。为此,我们提出Proto-Former,一种统一、自适应、端到端的人脸关键点检测框架,显式增强数据集特定的面部结构表示(即原型)。Proto-Former通过在统一架构中实现多数据集联合训练,克服了单数据集训练的局限性。具体而言,该框架包含两个核心组件:自适应原型感知编码器(APAE),用于自适应特征提取和原型表示学习;渐进式原型感知解码器(PPAD),用于优化原型以生成引导模型关注关键面部区域的提示。此外,我们引入一种新型原型感知(PA)损失,通过约束原型专家的选择权重,实现最优路径寻找。该损失有效解决了多数据集训练中的原型专家定位不稳定性问题,缓解梯度冲突,促进更精确的面部结构特征提取。大量实验表明,Proto-Former在广泛使用的基准数据集上优于现有最先进方法。代码已公开于:https://github.com/Husk021118/Proto-Former。

原文摘要 · Abstract (English)

Recent advances in deep learning have significantly improved facial landmark detection. However, existing facial landmark detection datasets often define different numbers of landmarks, and most mainstream methods can only be trained on a single dataset. This limits the model generalization to different datasets and hinders the development of a unified model. To address this issue, we propose Proto-Former, a unified, adaptive, end-to-end facial landmark detection framework that explicitly enhances dataset-specific facial structural representations (i.e., prototype). Proto-Former overcomes the limitations of single-dataset training by enabling joint training across multiple datasets within a unified architecture. Specifically, Proto-Former comprises two key components: an Adaptive Prototype-Aware Encoder (APAE) that performs adaptive feature extraction and learns prototype representations, and a Progressive Prototype-Aware Decoder (PPAD) that refines these prototypes to generate prompts that guide the model's attention to key facial regions. Furthermore, we introduce a novel Prototype-Aware (PA) loss, which achieves optimal path finding by constraining the selection weights of prototype experts. This loss function effectively resolves the problem of prototype expert addressing instability during multi-dataset training, alleviates gradient conflicts, and enables the extraction of more accurate facial structure features. Extensive experiments on widely used benchmark datasets demonstrate that our Proto-Former achieves superior performance compared to existing state-of-the-art methods. The code is publicly available at: https://github.com/Husk021118/Proto-Former.

人脸检测原型网络统一模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。