仅用头手三关节数据,逐步生成逼真全身动作。
MAGE:A Multi-stage Avatar Generator with Sparse Observations
- 分阶段渐进推断,从6大肢体块到22个关节
- 在大规模数据集上精度与连贯性超越现有方法
- 适合虚拟现实、动作捕捉等需低精度输入的场景
从仅捕捉头部和双手三个关节的头戴设备中推断完整身体姿态,是广泛应用于增强现实与虚拟现实的挑战性任务。以往方法多采用单阶段直接映射学习,导致未观测关节运动空间过大,常引发下肢预测不准与时间不连贯,生成动作不自然或逻辑矛盾。为此,本文提出多阶段化身生成器MAGE,将单一映射分解为渐进式预测策略:从6大肢体块开始,逐步细化至22个关节,每一步引入前序阶段的运动上下文先验,以更丰富约束降低不确定性。在大规模数据集上的大量实验表明,MAGE显著优于当前最优方法,在准确性和时序一致性方面均有提升。
原文摘要 · Abstract (English)
Inferring full-body poses from Head Mounted Devices, which capture only 3-joint observations from the head and wrists, is a challenging task with wide AR/VR applications. Previous attempts focus on learning one-stage motion mapping and thus suffer from an over-large inference space for unobserved body joint motions. This often leads to unsatisfactory lower-body predictions and poor temporal consistency, resulting in unrealistic or incoherent motion sequences. To address this, we propose a powerful Multi-stage Avatar GEnerator named MAGE that factorizes this one-stage direct motion mapping learning with a progressive prediction strategy. Specifically, given initial 3-joint motions, MAGE gradually inferring multi-scale body part poses at different abstract granularity levels, starting from a 6-part body representation and gradually refining to 22 joints. With decreasing abstract levels step by step, MAGE introduces more motion context priors from former prediction stages and thus improves realistic motion completion with richer constraint conditions and less ambiguity. Extensive experiments on large-scale datasets verify that MAGE significantly outperforms state-of-the-art methods with better accuracy and continuity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。