用金字塔结构建模人体长程依赖,提升3D姿态估计精度与效率
Learning Pyramid-structured Long-range Dependencies for 3D Human Pose Estimation
- 设计金字塔图注意力模块,跨尺度融合关节与肢体间的关联
- 在Human3.6M和MPI-INF-3DHP上误差更低,模型更轻量
- 适合需要高效高精度3D姿态估计的实时应用
人体动作协调对恢复3D姿态的空间约束至关重要,通常表现为身体部位间的长程依赖。然而,建模长程依赖面临两大挑战:一是关节不仅受其他关节约束,还受整体肢体调节;二是现有方法通过加深网络学习非连接部分的依赖,引入无关噪声并增大模型规模。本文提出一种金字塔结构,以更优方式捕捉关节与群体间的相关性,补全人体子结构上下文信息。通过创新的金字塔图注意力(PGA)模块,将多尺度信息融合为紧凑序列,并并行计算尺度间相关性。结合图卷积模块,构建轻量级多尺度变压器架构PGFormer,将人体子结构通过池化封装进自注意力机制。大量实验表明,该方法在Human3.6M和MPI-INF-3DHP数据集上均达到更低误差且模型更小。代码已开源。
原文摘要 · Abstract (English)
Action coordination in human structure is indispensable for the spatial constraints of 2D joints to recover 3D pose. Usually, action coordination is represented as a long-range dependence among body parts. However, there are two main challenges in modeling long-range dependencies. First, joints should not only be constrained by other individual joints but also be modulated by the body parts. Second, existing methods make networks deeper to learn dependencies between non-linked parts. They introduce uncorrelated noise and increase the model size. In this paper, we utilize a pyramid structure to better learn potential long-range dependencies. It can capture the correlation across joints and groups, which complements the context of the human sub-structure. In an effective cross-scale way, it captures the pyramid-structured long-range dependence. Specifically, we propose a novel Pyramid Graph Attention (PGA) module to capture long-range cross-scale dependencies. It concatenates information from various scales into a compact sequence, and then computes the correlation between scales in parallel. Combining PGA with graph convolution modules, we develop a Pyramid Graph Transformer (PGFormer) for 3D human pose estimation, which is a lightweight multi-scale transformer architecture. It encapsulates human sub-structures into self-attention by pooling. Extensive experiments show that our approach achieves lower error and smaller model size than state-of-the-art methods on Human3.6M and MPI-INF-3DHP datasets. The code is available at https://github.com/MingjieWe/PGFormer.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。