仅用头部姿态重建全身网格，速度比扩散模型快50倍以上。
EgoGVAE: Ego-body Mesh Reconstruction via Guided Variational Autoencoder
- 通过引导变分自编码器，将头部姿态映射到全身动作的潜在空间。
- 在基准数据集上实现更优的全身网格重建效果，且推理速度快于扩散模型50倍。
- 适合实时应用如智能眼镜、头戴设备，尤其看重速度与精度的场景。
本文解决仅从头部姿态恢复全身网格的问题，该任务对头戴设备和智能眼镜等应用至关重要。挑战在于仅凭单一关节（头部）轨迹推断未观测身体部位的姿态。现有方法多采用头部条件生成模型，但依赖基于扩散的迭代过程，计算成本高、耗时长。为此，我们提出一种简单而新颖的方法：利用引导网络的潜在空间，该网络为变分自编码器，以全身姿态作为输入。通过强制引导网络与头部到动作网络的潜在分布相似，可从“被引导”的分布中采样潜在特征，并可靠解码出仅基于头部姿态的自然全身姿态表示。关键优势在于，采用单步采样方案，推理速度比扩散方法提升超过50倍。在基准数据集上的实验表明，该方法显著提升了自我体网格重建性能。
原文摘要 · Abstract (English)
We address the problem of recovering the full-body mesh from only the head pose. This task has become essential for various applications based on head-mounted devices or smart glasses. The challenge of this task lies in estimating the pose information of unobserved body parts based solely on a single joint (i.e., head) trajectory. Several studies have begun to adopt head-conditioned generative models, however, such previous methods are costly and time-consuming due to the diffusion-based iterative process. As an alternative, we propose a simple yet novel method that leverages the latent space of the guidance network, which is designed as a variational autoencoder taking full-body poses as inputs. By enforcing latent distributions of this guidance network and our head-to-motion network to be similar, latent features sampled from the 'guided' distribution, i.e., distribution learned in our head-to-motion network, can be reliably decoded for natural representations of full-body poses even only with the head pose. One important advantage of the proposed method is that one-step sampling scheme achieves remarkably fast inference (more than 50 times faster) compared to diffusion-based approaches. Experimental results on benchmark datasets show that the proposed method efficiently improves the performance of ego-body mesh reconstruction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。