通过透视编码与旋转优化3D人体姿态估计,提升远距离姿势预测精度。
PersPose: 3D Human Pose Estimation with Perspective Encoding and Perspective Rotation
- 引入透视编码(PE)融合相机内参信息,增强深度感知。
- 设计透视旋转(PR)使人物居中,减少图像畸变,改善模型拟合。
- 在3DPW等数据集上达到当前最优,尤其对野外场景表现显著提升。
单目3D人体姿态估计从单张图像中推断关节的3D位置。现有方法通常仅以裁剪图像为输入,但缺乏相机内参时难以准确估计关节相对深度。本文提出透视编码(PE),用于编码裁剪图像的相机内参信息。由于人体可能出现在原始图像任意位置,导致3D场景与裁剪图像间的透视关系差异显著,加剧模型拟合难度;且人物离图像中心越远,裁剪图像中的透视畸变更严重。为此,我们提出透视旋转(PR),对原始图像进行变换以使人体居中,从而减轻透视畸变并降低建模难度。结合PE与PR,构建新框架PersPose。实验表明,PersPose在3DPW、MPI-INF-3DHP和Human3.6M数据集上均达到当前最优性能。例如,在野外数据集3DPW上,实现60.1 mm的MPJPE,比之前最优方法降低7.54%。代码已公开于:https://github.com/KenAdamsJoseph/PersPose。
原文摘要 · Abstract (English)
Monocular 3D human pose estimation (HPE) methods estimate the 3D positions of joints from individual images. Existing 3D HPE approaches often use the cropped image alone as input for their models. However, the relative depths of joints cannot be accurately estimated from cropped images without the corresponding camera intrinsics, which determine the perspective relationship between 3D objects and the cropped images. In this work, we introduce Perspective Encoding (PE) to encode the camera intrinsics of the cropped images. Moreover, since the human subject can appear anywhere within the original image, the perspective relationship between the 3D scene and the cropped image differs significantly, which complicates model fitting. Additionally, the further the human subject deviates from the image center, the greater the perspective distortions in the cropped image. To address these issues, we propose Perspective Rotation (PR), a transformation applied to the original image that centers the human subject, thereby reducing perspective distortions and alleviating the difficulty of model fitting. By incorporating PE and PR, we propose a novel 3D HPE framework, PersPose. Experimental results demonstrate that PersPose achieves state-of-the-art (SOTA) performance on the 3DPW, MPI-INF-3DHP, and Human3.6M datasets. For example, on the in-the-wild dataset 3DPW, PersPose achieves an MPJPE of 60.1 mm, 7.54% lower than the previous SOTA approach. Code is available at: https://github.com/KenAdamsJoseph/PersPose.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。