为手部关键点单独估计可见性,提升遮挡下姿态估计的可靠性。
Hand Visibility Detector: Per-Keypoint Visibility Estimation for Hands

- 基于预训练手姿模型,逐关键点预测可见性。
- 可见性加权三角测量使重投影误差降低。
- 首个将可见性估计作为独立任务的研究,适合高精度手部应用。
手部姿态估计(HPE)是AR/VR和机器人等应用的基础技术。在这些场景中,图像中每个手部关节的可见性对评估估计结果的可靠性至关重要。然而,现有多数HPE方法仅输出关节位置,未显式标注其可见性。尽管部分方法考虑了遮挡或可见性,但通常仅作为辅助信号用于提升姿态估计。据我们所知,针对单个手部关节的可见性估计尚未被系统研究为独立任务。本文提出Hand Visibility Detector,一个用于估计个体手部关节可见性的模型,并首次系统性地将可见性估计作为独立任务进行研究。我们证明,利用在大规模数据上预训练的HPE模型作为主干网络,可在该任务中取得优异性能。进一步实验表明,将可见性信息用于多视角2D关键点三角测量的3D手部姿态标注时,可见性加权的三角测量可有效降低重投影误差。该方法已开源,代码与演示见https://github.com/ryhara/hand_visibility_detector。
原文摘要 · Abstract (English)
Hand Pose Estimation (HPE) is a fundamental technology for various applications such as AR/VR and robotics. In these applications, the visibility of each hand joint in the image is crucial for assessing the reliability of estimation results under occlusion. However, most existing HPE methods output joint positions without explicitly indicating their visibility. Although some methods account for occlusion or visibility, visibility estimation has mainly been used as an auxiliary signal for improving pose estimation. To our knowledge, per-joint hand visibility estimation has not been systematically studied as a standalone task. In this work, we propose Hand Visibility Detector, a model for estimating the visibility of individual hand joints, and present the first systematic investigation of visibility estimation as an independent task. We show that leveraging the prior knowledge of HPE models pretrained on large-scale data as a backbone yields high performance in this task. We further demonstrate the utility of Hand Visibility Detector on a downstream task of 3D hand pose annotation via multi-view triangulation of 2D keypoints, showing that visibility-weighted triangulation reduces reprojection error. Our method is released as a ready-to-use package, and the code and demo are available at https://github.com/ryhara/hand_visibility_detector .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。