用可微高斯渲染实现无需3D标注的多视角人体姿态估计
SkelSplat: Robust Multi-view 3D Human Pose Estimation with Differentiable Gaussian Rendering
- 将人体关节建模为3D高斯点，通过可微渲染融合任意视角
- 在Human3.6M和CMU数据集上超越无3D标注方法，跨数据集误差降低47.8%
- 对遮挡鲁棒，无需针对特定场景微调，适合真实场景应用
准确的3D人体姿态估计对增强现实与人机交互至关重要。现有基于多视角的方法依赖大规模标注数据训练，导致测试场景变化时泛化能力差。为此，我们提出SkelSplat，一种基于可微高斯渲染的多视角3D人体姿态估计框架。人体姿态被建模为一组3D高斯点（每关节一个），通过可微渲染优化，在无3D真值监督下实现任意视角的无缝融合。由于高斯溅射原用于密集场景重建，我们设计了一种新颖的独热编码方案，实现人体关节的独立优化。SkelSplat在Human3.6M和CMU数据集上优于不依赖3D真值的方法，且相比学习型方法，跨数据集误差降低最高达47.8%。在Human3.6M-Occ和Occlusion-Person上的实验表明其对遮挡具有鲁棒性，无需场景特化微调。项目主页：https://skelsplat.github.io。
原文摘要 · Abstract (English)
Accurate 3D human pose estimation is fundamental for applications such as augmented reality and human-robot interaction. State-of-the-art multi-view methods learn to fuse predictions across views by training on large annotated datasets, leading to poor generalization when the test scenario differs. To overcome these limitations, we propose SkelSplat, a novel framework for multi-view 3D human pose estimation based on differentiable Gaussian rendering. Human pose is modeled as a skeleton of 3D Gaussians, one per joint, optimized via differentiable rendering to enable seamless fusion of arbitrary camera views without 3D ground-truth supervision. Since Gaussian Splatting was originally designed for dense scene reconstruction, we propose a novel one-hot encoding scheme that enables independent optimization of human joints. SkelSplat outperforms approaches that do not rely on 3D ground truth in Human3.6M and CMU, while reducing the cross-dataset error up to 47.8% compared to learning-based methods. Experiments on Human3.6M-Occ and Occlusion-Person demonstrate robustness to occlusions, without scenario-specific fine-tuning. Our project page is available here: https://skelsplat.github.io.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。