arXiv:2604.17688cs.CV2026-04被引 17

融合局部骨骼关系与全局时空信息,提升3D人体姿态估计精度

Dual-stream Spatio-Temporal GCN-Transformer Network for 3D Human Pose Estimation

  • 双流结构并行建模空间与时间关系,结合GCN与Transformer优势
  • 在Human3.6M和MPI-INF-3DHP上分别实现37.6mm和15.7mm的P1误差
  • 适合关注动作捕捉、人机交互等应用的研究者

3D人体姿态估计是计算机视觉中的经典重要方向。近年来,基于Transformer的方法在从2D到3D人体姿态提升方面取得显著进展。然而,这些方法主要关注全局时空关系建模,忽视了局部骨骼关系及不同通道间的信息交互。为此,本文提出一种新方法——双流时空图卷积网络-变压器混合模型(MixTGFormer)。该方法通过两条并行路径同时建模人体骨骼的空间与时间关系,实现全局与局部特征的有效融合。MixTGFormer的核心由堆叠的Mixformer构成,包含混合块(Mixformer Block)与压缩-激励层(SE Layer)。首先,两个具有不同模式的Mixformer Block并行提取并融合人体骨骼的多维信息;随后,SE Layer进一步补充融合后的信息。其中,混合块将图卷积网络(GCN)融入Transformer,增强局部与全局信息利用。此外,分别实现其时空形式以提取空间与时间关系。我们在两个基准数据集(Human3.6M 和 MPI-INF-3DHP)上进行了广泛评估。实验结果表明,相较于其他方法,MixTGFormer在两项数据集上分别达到37.6mm和15.7mm的P1误差,达到当前最优水平。

原文摘要 · Abstract (English)

3D human pose estimation is a classic and important research direction in the field of computer vision. In recent years, Transformer-based methods have made significant progress in lifting 2D to 3D human pose estimation. However, these methods primarily focus on modeling global temporal and spatial relationships, neglecting local skeletal relationships and the information interaction between different channels. Therefore, we have proposed a novel method,the Dual-stream Spatio-temporal GCN-Transformer Network (MixTGFormer). This method models the spatial and temporal relationships of human skeletons simultaneously through two parallel channels, achieving effective fusion of global and local features. The core of MixTGFormer is composed of stacked Mixformers. Specifically, the Mixformer includes the Mixformer Block and the Squeeze-and-Excitation Layer ( SE Layer). It first extracts and fuses various information of human skeletons through two parallel Mixformer Blocks with different modes. Then, it further supplements the fused information through the SE Layer. The Mixformer Block integrates Graph Convolutional Networks (GCN) into the Transformer, enhancing both local and global information utilization. Additionally, we further implement its temporal and spatial forms to extract both spatial and temporal relationships. We extensively evaluated our model on two benchmark datasets (Human3.6M and MPI-INF-3DHP). The experimental results showed that, compared to other methods, our MixTGFormer achieved state-of-the-art results, with P1 errors of 37.6mm and 15.7mm on these datasets, respectively.

3D姿态估计图神经网络Transformer人体动作分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。