arXiv:2503.22359cs.CV2025-03IJCV被引 3

统一多数据集人脸对齐知识,提升模型泛化与少样本性能。

Mitigating Knowledge Discrepancies among Multiple Datasets for Task-agnostic Unified Face Alignment

  • 用均值人脸和语义嵌入对齐不同数据集的标注结构。
  • 在7个基准上显著提升少样本与零样本人脸对齐效果。
  • 可零样本定位未见关键点,适合跨数据集迁移任务。

尽管人脸结构相似,现有方法难以从标注不同的多数据集中学习统一知识,且单个数据集样本有限导致模型鲁棒性差。本文提出一种统一多数据集知识的策略,为每个数据集计算均值人脸形状,并通过语义对齐嵌入将其投影到可解释平面上,形成坐标锚点。将这些锚点编码为结构提示,结合图像特征回归对应的关键点,建立从平面到目标人脸的映射,统一各数据集的学习目标。该方法有效缓解了知识差异,提升模型泛化能力,显著增强向新数据集的知识迁移效率,在少样本人脸对齐中表现优异。此外,可解释平面赋予模型任务无关特性,实现零样本定位训练中未见的关键点。在7个基准上的实验验证了其显著性能提升。代码已开源。

原文摘要 · Abstract (English)

Despite the similar structures of human faces, existing face alignment methods cannot learn unified knowledge from multiple datasets with different landmark annotations. The limited training samples in a single dataset commonly result in fragile robustness in this field. To mitigate knowledge discrepancies among different datasets and train a task-agnostic unified face alignment (TUFA) framework, this paper presents a strategy to unify knowledge from multiple datasets. Specifically, we calculate a mean face shape for each dataset. To explicitly align these mean shapes on an interpretable plane based on their semantics, each shape is then incorporated with a group of semantic alignment embeddings. The 2D coordinates of these aligned shapes can be viewed as the anchors of the plane. By encoding them into structure prompts and further regressing the corresponding facial landmarks using image features, a mapping from the plane to the target faces is finally established, which unifies the learning target of different datasets. Consequently, multiple datasets can be utilized to boost the generalization ability of the model. The successful mitigation of discrepancies also enhances the efficiency of knowledge transferring to a novel dataset, significantly boosts the performance of few-shot face alignment. Additionally, the interpretable plane endows TUFA with a task-agnostic characteristic, enabling it to locate landmarks unseen during training in a zero-shot manner. Extensive experiments are carried on seven benchmarks and the results demonstrate an impressive improvement in face alignment brought by knowledge discrepancies mitigation. The code is available at https://github.com/Jiahao-UTS/TUFA.

人脸对齐少样本学习知识迁移任务无关

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。