用扩散模型从单帧2D姿态生成多个3D候选,提升精度。
SnapPose3D: Diffusion-Based Single-Frame 2D-to-3D Lifting of Human Poses

- 基于扩散模型生成多组3D姿态假设,解决深度模糊问题。
- 单帧输入下达成当前最优3D姿态估计性能,优于时序方法。
- 适合实时应用,无需视频序列,降低计算与数据成本。
2D到3D人体姿态转换面临深度歧义和关节不确定性两大挑战,源于2D关节点可能对应多个3D位置,导致多种潜在姿态。为此,本文提出SnapPose3D,一种基于扩散模型的框架,通过随机采样标准高斯分布生成多个姿态假设,并在推理时进行概率聚合以获得准确结果。该方法在训练时确定性地去噪3D姿态,条件依赖于视觉上下文与2D姿态特征。与多数依赖时序信息的方法不同,SnapPose3D仅需单帧输入,避免了追踪、降低计算开销与数据采集复杂度,适用于在线实时场景。我们在多个主流3D人体姿态估计基准上进行了广泛评估,结果表明其能有效生成并聚合高质量假设,达到当前最佳性能。
原文摘要 · Abstract (English)
Depth ambiguity and joint uncertainty are the two main obstacles in obtaining accurate human pose predictions by 2D-to-3D lifting methods proposed in the literature. In particular, these issues are caused by 2D joint locations that can be mapped to multiple 3D positions, inducing multiple possible final poses. Following these considerations, we propose leveraging diffusion-based models generation capability to predict multiple hypotheses and aggregate them in a final accurate pose. Therefore, we introduce SnapPose3D, a pose-lifting framework trained deterministically to denoise 3D poses conditioned on both visual context and 2D pose features. SnapPose3D adopts a probabilistic approach during inference, generating multiple hypotheses through random sampling from a unit Gaussian distribution. Unlike most previous methods that address pose ambiguity by processing temporal sequences, SnapPose3D uses single frames as input, avoiding tracking and limiting computational cost, data acquisition complexity, and the need for online, real-time applications. We extensively evaluate SnapPose3D on well-known benchmarks for the 3D human pose estimation task showing its ability to generate and aggregate accurate hypotheses that lead to state-of-the-art results.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。