用神经距离场建模人体动作合理性，提升三维动作估计精度
MoManifold: Learning to Measure 3D Human Motion via Decoupled Joint Acceleration Manifolds
- 基于神经距离场构建解耦关节加速度流形，显式量化动作合理性
- 在去噪、补全、降抖等任务中优于现有最优方法，最高提升12.3%
- 适合需要高保真动作生成与修复的AR/VR、动画制作场景
有效融入时间信息对精准的3D人体动作估计与生成至关重要，广泛应用于人机交互和AR/VR等领域。本文提出MoManifold，一种新型人体动作先验，可在连续高维动作空间中建模合理的人体运动。不同于传统数学或变分自编码器方法，其表示基于神经距离场，将人体动力学显式量化为可度量的分数，从而评估动作合理性。我们提出解耦关节加速度流形，从有限动作数据中建模人体动态。此外，引入以流形距离为引导的新优化方法，适用于多种动作相关任务。大量实验表明，作为先验，MoManifold在多个下游任务中超越现有SOTA，包括真实动作捕捉数据去噪、部分3D观测下的动作恢复、基于SMPL的姿态估计算法降抖，以及动作插值结果的优化。
原文摘要 · Abstract (English)
Incorporating temporal information effectively is important for accurate 3D human motion estimation and generation which have wide applications from human-computer interaction to AR/VR. In this paper, we present MoManifold, a novel human motion prior, which models plausible human motion in continuous high-dimensional motion space. Different from existing mathematical or VAE-based methods, our representation is designed based on the neural distance field, which makes human dynamics explicitly quantified to a score and thus can measure human motion plausibility. Specifically, we propose novel decoupled joint acceleration manifolds to model human dynamics from existing limited motion data. Moreover, we introduce a novel optimization method using the manifold distance as guidance, which facilitates a variety of motion-related tasks. Extensive experiments demonstrate that MoManifold outperforms existing SOTAs as a prior in several downstream tasks such as denoising real-world human mocap data, recovering human motion from partial 3D observations, mitigating jitters for SMPL-based pose estimators, and refining the results of motion in-betweening.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。