通过双流结构融合局部与全局依赖,提升3D人体姿态估计精度。
Optimizing Local-Global Dependencies for Accurate 3D Human Pose Estimation
- 设计SSRFormer模块,用骨架选择性精炼注意力捕捉局部细节
- 在Human3.6M和MPI-INF-3DHP上分别达37.4mm和13.2mm的P1误差
- 适合需要高精度姿态建模的下游任务,如人体网格重建
基于Transformer的方法在3D人体姿态估计中取得显著进展,因其强大的长程依赖建模能力。然而,仅依赖全局注意力机制难以捕捉对准确估计至关重要的细粒度局部细节。为此,我们提出SSR-STF双流模型,有效融合局部特征与全局依赖以增强3D人体姿态估计。具体而言,引入SSRFormer模块,采用骨架选择性精炼注意力(SSRA)机制,捕获人体姿态序列中的细粒度局部依赖,弥补Transformer对全局依赖的建模。通过自适应融合两条特征流,SSR-STF能更好学习人体姿态的内在结构,克服传统方法在局部特征提取上的局限。在Human3.6M和MPI-INF-3DHP数据集上的大量实验表明,该模型达到当前最优性能,P1误差分别为37.4mm和13.2mm,同时在泛化能力与准确性上均优于现有方法。此外,模型学习到的运动表示在人体网格恢复等下游任务中也表现出色。代码已公开于https://github.com/poker-xu/SSR-STF。
原文摘要 · Abstract (English)
Transformer-based methods have recently achieved significant success in 3D human pose estimation, owing to their strong ability to model long-range dependencies. However, relying solely on the global attention mechanism is insufficient for capturing the fine-grained local details, which are crucial for accurate pose estimation. To address this, we propose SSR-STF, a dual-stream model that effectively integrates local features with global dependencies to enhance 3D human pose estimation. Specifically, we introduce SSRFormer, a simple yet effective module that employs the skeleton selective refine attention (SSRA) mechanism to capture fine-grained local dependencies in human pose sequences, complementing the global dependencies modeled by the Transformer. By adaptively fusing these two feature streams, SSR-STF can better learn the underlying structure of human poses, overcoming the limitations of traditional methods in local feature extraction. Extensive experiments on the Human3.6M and MPI-INF-3DHP datasets demonstrate that SSR-STF achieves state-of-the-art performance, with P1 errors of 37.4 mm and 13.2 mm respectively, outperforming existing methods in both accuracy and generalization. Furthermore, the motion representations learned by our model prove effective in downstream tasks such as human mesh recovery. Codes are available at https://github.com/poker-xu/SSR-STF.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。