提出抗遮挡的3D手物姿态估计方法，提升遮挡下关键点识别准确率
OccRobNet : Occlusion Robust Network for Accurate 3D Interacting Hand-Object Pose Estimation
- 先用CNN定位关节，再通过自注意力与交叉注意力融合上下文信息
- 在InterHand2.6M等3个数据集上达到当前最优性能
- 特别适合处理手部遮挡或双手交互场景的精准姿态估计
遮挡是3D手部姿态估计中的重大挑战，尤其在手物交互或双手协同时更为突出。现有方法较少关注被遮挡区域，而这些区域其实包含对姿态估计至关重要的信息。本文提出一种抗遮挡的3D手物姿态估计方法，首先利用基于CNN的模型定位手部关节，随后通过提取上下文信息进行精修。自注意力变换器识别特定关节及其所属手的身份，从而帮助在遮挡区域中仍能准确判断关节归属。进一步，结合关节身份信息与交叉注意力机制实现姿态估计。该方法在InterHand2.6M、HO3D和H$_2$O3D三个数据集上均取得当前最优结果，显著提升了遮挡条件下的估计鲁棒性。
原文摘要 · Abstract (English)
Occlusion is one of the challenging issues when estimating 3D hand pose. This problem becomes more prominent when hand interacts with an object or two hands are involved. In the past works, much attention has not been given to these occluded regions. But these regions contain important and beneficial information that is vital for 3D hand pose estimation. Thus, in this paper, we propose an occlusion robust and accurate method for the estimation of 3D hand-object pose from the input RGB image. Our method includes first localising the hand joints using a CNN based model and then refining them by extracting contextual information. The self attention transformer then identifies the specific joints along with the hand identity. This helps the model to identify the hand belongingness of a particular joint which helps to detect the joint even in the occluded region. Further, these joints with hand identity are then used to estimate the pose using cross attention mechanism. Thus, by identifying the joints in the occluded region, the obtained network becomes robust to occlusion. Hence, this network achieves state-of-the-art results when evaluated on the InterHand2.6M, HO3D and H$_2$O3D datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。