统一动态人脸关键点检测,一模型适配任意数量关键点
Towards Unified Dynamic Face Landmark Detection

- 用部位锚定位置表示法统一不同数据集的关键点
- 单模型可动态输出任意数量的关键点,无需重新训练
- 适合需要灵活调整关键点数量的应用场景
尽管人脸关键点检测方法持续提升性能,但仍存在两大功能局限:(1) 每个‘N点’基准数据集需独立训练网络参数;(2) 在‘N点’数据集上训练的模型仅能可靠输出N个关键点。本文提出人脸部位锚定关键点位置(FPALPs),将每个关键点视为面部区域轮廓上的0到1之间连续值,使所有‘N点’数据集可统一为单一数据集。同时,基于FPALP的查询向量经跨模态解码器逐步优化,最终预测坐标。所提统一动态FLD方法实现:(1) 单一模型可学习任意数量‘N点’数据集;(2) 运行时加载指定关键点查询即可输出任意数量关键点。在多个基准数据集上的实验表明,该方法在保持竞争力的同时,在部分任务中超越现有最优方法。
原文摘要 · Abstract (English)
Although advancements in face landmark detection (FLD) methods continue to push performance boundaries, they overlook two major functional limitations: (1) different network parameters need to be trained independently for each ``$N$-point'' benchmark dataset, and (2) a model trained on an ``$N$-point'' dataset reliably outputs only the $N$ landmarks. In our work, we first conceptualize Face Part-Anchored Landmark Positions (FPALPs), wherein each landmark is treated as a progression value between zero (start) and one (end) along a face part's contour. Every landmark can be expressed in the FPALP format, irrespective of its source dataset, hence unlocking the ability to unify all ``$N$-point'' datasets into a single dataset. Secondly, we represent each landmark with an FPALP-based query, refine it progressively with a cross-modality decoder, and predict its coordinates based on the final representation. Our approach, called Unified Dynamic FLD, embodies these two design choices and streamlines the landmark detection pipeline by enabling (1) a single model to learn on any number of ``$N$-point'' datasets, and (2) yield any number of specific landmark predictions by loading the designated landmark queries at runtime. Extensive experiments on multiple benchmark datasets show that our method delivers these benefits while remaining competitive with, and in several cases outperforming existing state-of-the-art methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。