用2D扩散模型先验实现单视角3D人体姿态无监督估计
Unsupervised 3D Human Pose Estimation via Conditional Multi-view Ancestral Sampling
- 基于2D扩散模型先验，设计条件多视角采样方法优化3D姿态
- 在Yoga数据集上优于主流有监督与无监督方法，包括极端姿势
- 适合缺乏3D标注但需高精度3D姿态的场景，如动作捕捉
我们提出一种无需3D监督即可从单视角估计3D人体姿态的方法。核心是利用在大规模2D人体姿态数据集上预训练的运动扩散模型（MDMs）的2D扩散先验。具体地，将扩散模型的多视角祖先采样扩展至2D到3D姿态提升任务。为此，我们提出新的条件多视角祖先采样（cMAS），通过优化3D姿态使其多视角投影落在2D MDM噪声空间流形上，同时约束3D姿态匹配给定2D姿态和人体解剖约束。在Yoga数据集上的实验表明，该方法在跨域性能上优于当前最先进的有监督和无监督3D姿态估计方法，尤其在缺乏3D标注的极端人体姿态上表现更优。代码已公开于：https://github.com/asaa0001/c-MAS。
原文摘要 · Abstract (English)
We propose a method of estimating a 3D human pose from a single view without 3D supervision. The key to our method is to leverage the 2D diffusion priors of motion diffusion models (MDMs) pre-trained on large 2D human pose datasets. Specifically, we extend multi-view ancestral sampling of diffusion models to the task of 2D-3D lifting of human pose. To this end, we newly propose a conditional multi-view ancestral sampling (cMAS) that optimizes the 3D pose such that its multi-view projections follow the manifold in 2D MDM noise space, while conditioning the 3D pose to match the given 2D poses and anatomical constraints of humans. Experiments on the Yoga dataset demonstrate that our method achieves better cross-domain performance compared to state-of-the-art supervised and unsupervised 3D pose estimation methods, including extreme human poses where 3D supervision is unavailable. Code is available at: https://github.com/asaa0001/c-MAS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。