用相对姿态预测提升头姿估计精度,无需真实数据训练
VGGT-HPE: Reframing Head Pose Estimation as Relative Pose Prediction

- 将头姿估计转为相对位姿预测,避免依赖隐式参考帧
- 零真实数据训练下,在BIWI基准上超越主流方法
- 可动态选择参考帧,适合对精度要求高的实际应用
单目头姿估计传统上是将图像直接回归到绝对姿态,迫使网络隐式学习数据集特定的参考坐标系。本文提出,预测两个头部姿态间的相对刚性变换更本质且更鲁棒。我们构建了基于通用几何基础模型的VGGT-HPE,仅在合成人脸渲染数据上微调,通过显式提供已知姿态的锚点,将问题转化为估计几何位移。这一设计使测试时可灵活选择锚点(如近中性姿态或相邻帧),从而控制预测难度。尽管未使用任何真实世界数据,VGGT-HPE在BIWI基准上达到当前最优性能,显著优于在混合及真实数据上训练的绝对回归方法。通过可控的易/难样本对实验,系统验证了相对预测在准确性上天然优于绝对回归,优势随目标姿态难度增加而增强。
原文摘要 · Abstract (English)
Monocular head pose estimation is traditionally formulated as direct regression from a single image to an absolute pose. This paradigm forces the network to implicitly internalize a dataset-specific canonical reference frame. In this work, we argue that predicting the relative rigid transformation between two observed head configurations is a fundamentally easier and more robust formulation. We introduce VGGT-HPE, a relative head pose estimator built upon a general-purpose geometry foundation model. Finetuned exclusively on synthetic facial renderings, our method sidesteps the need for an implicit anchor by reducing the problem to estimating a geometric displacement from an explicitly provided anchor with a known pose. As a practical benefit, the relative formulation also allows the anchor to be chosen at test time - for instance, a near-neutral frame or a temporally adjacent one - so that the prediction difficulty can be controlled by the application. Despite zero real-world training data, VGGT-HPE achieves state-of-the-art results on the BIWI benchmark, outperforming established absolute regression methods trained on mixed and real datasets. Through controlled easy- and hard-pair benchmarks, we also systematically validate our core hypothesis: relative prediction is intrinsically more accurate than absolute regression, with the advantage scaling alongside the difficulty of the target pose. Project page and code: https://vasilikivas.github.io/VGGT-HPE
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。