用点云序列的时序信息提升3D人体姿态估计精度
SPiKE: 3D Human Pose from Point Cloud Sequences
- 通过Transformer捕捉点云序列中时空关联特征
- 在ITOP数据集上达89.19% mAP，推理更快
- 适合需要高效高精度姿态估计的应用场景
3D人体姿态估计（HPE）旨在从2D或3D输入（如RGB图像、深度图或点云）中定位人体关键点的3D位置。当前基于深度图和点云的方法大多依赖单帧估计，未利用序列中的时序信息。本文提出SPiKE，一种基于点云序列的新型3D HPE方法。不同于独立处理每帧的现有方法，SPiKE采用Transformer架构，编码跨序列点之间的时空关系。通过将点云划分为局部体素，并使用点空间卷积进行空间特征提取，SPiKE在保持每时刻空间完整性的同时，确保Transformer的高效处理。在ITOP基准测试中，SPiKE达到89.19% mAP，性能领先且推理时间显著降低。大量消融实验进一步验证了序列利用及算法设计的有效性。代码与模型已公开于：https://github.com/iballester/SPiKE
原文摘要 · Abstract (English)
3D Human Pose Estimation (HPE) is the task of locating keypoints of the human body in 3D space from 2D or 3D representations such as RGB images, depth maps or point clouds. Current HPE methods from depth and point clouds predominantly rely on single-frame estimation and do not exploit temporal information from sequences. This paper presents SPiKE, a novel approach to 3D HPE using point cloud sequences. Unlike existing methods that process frames of a sequence independently, SPiKE leverages temporal context by adopting a Transformer architecture to encode spatio-temporal relationships between points across the sequence. By partitioning the point cloud into local volumes and using spatial feature extraction via point spatial convolution, SPiKE ensures efficient processing by the Transformer while preserving spatial integrity per timestamp. Experiments on the ITOP benchmark for 3D HPE show that SPiKE reaches 89.19% mAP, achieving state-of-the-art performance with significantly lower inference times. Extensive ablations further validate the effectiveness of sequence exploitation and our algorithmic choices. Code and models are available at: https://github.com/iballester/SPiKE
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。