arXiv:2503.23331cs.CVcs.LG2025-03CVPR被引 17

通过分层生成2D密集姿态,提升遮挡下3D人体姿态估计性能。

HiPART: Hierarchical Pose AutoRegressive Transformer for Occluded 3D Human Pose Estimation

  • 先将稀疏2D骨架转为分层令牌,再分层自回归生成密集姿态。
  • 在单帧3D姿态估计上达到最新性能,优于多数多帧方法。
  • 降低参数与计算量,适合资源受限场景或作为增强模块使用。

现有2D到3D人体姿态估计方法在提升阶段依赖时序和视觉线索来缓解遮挡问题,但忽略了稀疏骨架2D输入表示的根本局限性,这反而加剧了遮挡影响。为此,本文提出一种两阶段生成式稠化方法——分层姿态自回归变压器(HiPART),从原始稀疏2D姿态生成分层密集姿态。首先,设计多尺度骨架令牌化模块,将高密度2D姿态量化为分层令牌,并引入骨架感知对齐以强化令牌间关联;随后,构建分层自回归建模机制实现分层2D姿态生成。以生成的分层姿态作为2D到3D提升的输入,所提方法在单帧3D人体姿态估计中展现出强鲁棒性,在多个基准上达到当前最优性能。同时,其参数量与计算开销更低,优于众多多帧方法,并可与多帧方法结合进一步提升性能与鲁棒性。

原文摘要 · Abstract (English)

Existing 2D-to-3D human pose estimation (HPE) methods struggle with the occlusion issue by enriching information like temporal and visual cues in the lifting stage. In this paper, we argue that these methods ignore the limitation of the sparse skeleton 2D input representation, which fundamentally restricts the 2D-to-3D lifting and worsens the occlusion issue. To address these, we propose a novel two-stage generative densification method, named Hierarchical Pose AutoRegressive Transformer (HiPART), to generate hierarchical 2D dense poses from the original sparse 2D pose. Specifically, we first develop a multi-scale skeleton tokenization module to quantize the highly dense 2D pose into hierarchical tokens and propose a Skeleton-aware Alignment to strengthen token connections. We then develop a Hierarchical AutoRegressive Modeling scheme for hierarchical 2D pose generation. With generated hierarchical poses as inputs for 2D-to-3D lifting, the proposed method shows strong robustness in occluded scenarios and achieves state-of-the-art performance on the single-frame-based 3D HPE. Moreover, it outperforms numerous multi-frame methods while reducing parameter and computational complexity and can also complement them to further enhance performance and robustness.

3D姿态估计遮挡处理生成模型自回归

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。