用运动学引导的模型，从头戴设备稀疏信号中高效还原自然人体动作。
KineST: A Kinematics-guided Spatiotemporal State Space Model for Human Motion Tracking from Sparse Signals
- 引入运动学先验的双向扫描机制，捕捉关节间复杂关系。
- 轻量级框架下实现高精度与动作连贯性的平衡，误差低于现有方法。
- 适合需要实时性与稳定性的虚拟现实场景应用。
全身动作追踪在AR/VR应用中至关重要，连接物理世界与虚拟交互。然而，仅凭头戴设备获取的稀疏信号重建真实且多样的全身姿态仍具挑战。现有方法常伴随高计算开销或分步建模时空依赖，难以兼顾精度、时序连贯性与效率。为此，我们提出KineST——一种基于运动学引导的状态空间模型，有效提取时空依赖并融合局部与全局姿态感知。核心创新包括：一、将状态空间对偶框架中的扫描策略重构为运动学引导的双向扫描，嵌入运动学先验；二、采用混合时空表示学习方法，紧密耦合空间与时间上下文，平衡精度与平滑性；三、引入几何角速度损失，对旋转变化施加物理合理约束，提升运动稳定性。大量实验表明，KineST在轻量级架构下显著优于现有方法，在准确性和时序一致性上均表现优异。
原文摘要 · Abstract (English)
Full-body motion tracking plays an essential role in AR/VR applications, bridging physical and virtual interactions. However, it is challenging to reconstruct realistic and diverse full-body poses based on sparse signals obtained by head-mounted displays, which are the main devices in AR/VR scenarios. Existing methods for pose reconstruction often incur high computational costs or rely on separately modeling spatial and temporal dependencies, making it difficult to balance accuracy, temporal coherence, and efficiency. To address this problem, we propose KineST, a novel kinematics-guided state space model, which effectively extracts spatiotemporal dependencies while integrating local and global pose perception. The innovation comes from two core ideas. Firstly, in order to better capture intricate joint relationships, the scanning strategy within the State Space Duality framework is reformulated into kinematics-guided bidirectional scanning, which embeds kinematic priors. Secondly, a mixed spatiotemporal representation learning approach is employed to tightly couple spatial and temporal contexts, balancing accuracy and smoothness. Additionally, a geometric angular velocity loss is introduced to impose physically meaningful constraints on rotational variations for further improving motion stability. Extensive experiments demonstrate that KineST has superior performance in both accuracy and temporal consistency within a lightweight framework. Project page: https://kaka-1314.github.io/KineST/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。