轻量级3D人体姿态估计模型,适配边缘设备部署
NanoHTNet: Nano Human Topology Network for Efficient 3D Human Pose Estimation
- 基于人体拓扑结构设计分层混合网络,捕捉多层级空间特征
- 结合离散余弦变换与低通滤波,有效建模局部动作与全局连贯性
- 通过对比学习预训练提升特征表达,显著降低计算开销
3D人体姿态估计在资源受限的边缘设备上应用受限,亟需更高效的模型。本文提出纳米级人体拓扑网络(NanoHTNet),采用堆叠的分层混合器结构,其中空间分层混合器高效学习人体物理拓扑在多语义层次上的特征,时间分层混合器结合离散余弦变换与低通滤波,捕捉局部瞬时运动和全局动作连贯性。此外,提出高效的时空标记化(ETST)方法,显著增强时空交互并降低计算复杂度。进一步提出PoseCLR,一种基于对比学习的通用预训练方法,通过在不同视角的2D姿态间对齐,帮助3D HPE编码器提取人体拓扑的隐式表示,提升高维特征表达能力。大量实验表明,结合PoseCLR的NanoHTNet在效率上优于现有最优方法,非常适合部署于Jetson Nano等边缘设备。代码与模型已开源。
原文摘要 · Abstract (English)
The widespread application of 3D human pose estimation (HPE) is limited by resource-constrained edge devices, requiring more efficient models. A key approach to enhancing efficiency involves designing networks based on the structural characteristics of input data. However, effectively utilizing the structural priors in human skeletal inputs remains challenging. To address this, we leverage both explicit and implicit spatio-temporal priors of the human body through innovative model design and a pre-training proxy task. First, we propose a Nano Human Topology Network (NanoHTNet), a tiny 3D HPE network with stacked Hierarchical Mixers to capture explicit features. Specifically, the spatial Hierarchical Mixer efficiently learns the human physical topology across multiple semantic levels, while the temporal Hierarchical Mixer with discrete cosine transform and low-pass filtering captures local instantaneous movements and global action coherence. Moreover, Efficient Temporal-Spatial Tokenization (ETST) is introduced to enhance spatio-temporal interaction and reduce computational complexity significantly. Second, PoseCLR is proposed as a general pre-training method based on contrastive learning for 3D HPE, aimed at extracting implicit representations of human topology. By aligning 2D poses from diverse viewpoints in the proxy task, PoseCLR aids 3D HPE encoders like NanoHTNet in more effectively capturing the high-dimensional features of the human body, leading to further performance improvements. Extensive experiments verify that NanoHTNet with PoseCLR outperforms other state-of-the-art methods in efficiency, making it ideal for deployment on edge devices like the Jetson Nano. Code and models are available at https://github.com/vefalun/NanoHTNet.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。