通过动态关注关节图阶次,提升3D人体姿态估计精度。
3D Human Pose Estimation via Spatial Graph Order Attention and Temporal Body Aware Transformer
- 用多阶图建模骨骼空间关系,结合图阶注意力自适应选择关键结构。
- 在三个主流数据集上优于现有方法,尤其在复杂动作上表现更优。
- 适合需要高精度姿态估计的动画、医疗和体育分析场景。
当前3D人体姿态估计主要依赖Transformer与图卷积网络(GCN)。然而,基于Transformer的方法或忽略关节间的空间邻近关系,或忽视局部关节运动的时间模式;而基于GCN的方法常缺乏针对姿态特性的表示能力。为此,本文提出一种新方法:利用GCN为每个骨架构建不同阶次的多图表示,并引入图阶注意力模块,动态强调每个关节最具有代表性的图阶。序列的空间特征经由提出的时序身体感知变压器处理,该模型在捕捉全局身体特征依赖的同时,关注关节间的局部特征关联。由于输出姿态对齐于序列中心2D姿态,我们改进了自注意力机制,使其在中心姿态处保持关注,逐步减弱对首尾姿态的关注。在Human3.6m、MPIINF-3DHP和HumanEva-I数据集上的大量实验验证了该方法的有效性。代码与模型已开源。
原文摘要 · Abstract (English)
Nowadays, Transformers and Graph Convolutional Networks (GCNs) are the prevailing techniques for 3D human pose estimation. However, Transformer-based methods either ignore the spatial neighborhood relationships between the joints when used for skeleton representations or disregard the local temporal patterns of the local joint movements in skeleton sequence modeling, while GCN-based methods often neglect the need for pose-specific representations. To address these problems, we propose a new method that exploits the graph modeling capability of GCN to represent each skeleton with multiple graphs of different orders, incorporated with a newly introduced Graph Order Attention module that dynamically emphasizes the most representative orders for each joint. The resulting spatial features of the sequence are further processed using a proposed temporal Body Aware Transformer that models the global body feature dependencies in the sequence with awareness of the local inter-skeleton feature dependencies of joints. Given that our 3D pose output aligns with the central 2D pose in the sequence, we improve the self-attention mechanism to be aware of the central pose while diminishing its focus gradually towards the first and the last poses. Extensive experiments on Human3.6m, MPIINF-3DHP, and HumanEva-I datasets demonstrate the effectiveness of the proposed method. Code and models are made available on Github.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。