用新框架提升3D人体姿态估计,兼顾骨骼结构与运动差异。
A Structure-aware and Motion-adaptive Framework for 3D Human Pose Estimation with Mamba

- 分两模块:一个建模骨骼拓扑,一个识别关节运动特性。
- 在多个基准上表现领先,计算开销更小。
- 适合需要精准动态建模的动画、动作分析场景。
当前基于Mamba的姿势提升方法多通过2D到1D映射及不同扫描策略建模关节依赖关系,虽有效但难以捕捉复杂关节连接,且对所有关节运动轨迹处理方式单一,忽视运动特性的内在差异。本文提出结构感知与运动自适应框架SAMA,包含结构感知状态积分器(SSI)和运动自适应状态调制器(MSM)。SSI基于姿态拓扑而非序列状态转移,在特征与状态层面融合关节间动态关系;MSM则识别各关节的特定运动特征,对不同运动模式进行定制化调整。通过上述模块,算法实现结构感知与运动自适应的姿态提升。大量实验表明,该方法在多个基准上达到先进性能,同时计算成本更低。
原文摘要 · Abstract (English)
Recent Mamba-based methods for the pose-lifting task tend to model joint dependencies by 2D-to-1D mapping with diverse scanning strategies. Though effective, they struggle to model intricate joint connections and uniformly process all joint motion trajectories while neglecting the intrinsic differences across motion characteristics. In this work, we propose a structure-aware and motion-adaptive framework to capture spatial joint topology along with diverse motion dynamics independently, named as SAMA. Specifically, SAMA consists of a Structure-aware State Integrator (SSI) and a Motion-adaptive State Modulator (MSM). The Structure-aware State Integrator is tasked with leveraging dynamic joint relationships to fuse information at both the joint feature and state levels in the state space, based on pose topology rather than sequential state transitions. The Motion-adaptive State Modulator is responsible for joint-specific motion characteristics recognition, thus applying tailored adjustments to diverse motion patterns across different joints. Through the above key modules, our algorithm enables structure-aware and motion-adaptive pose lifting. Extensive experiments across multiple benchmarks demonstrate that our algorithm achieves advanced results with fewer computational costs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。