arXiv:2411.03086cs.CVcs.AI2024-11被引 3

实时重建人体3D模型并同步输出骨骼与姿态,兼顾精度与效率。

HFGaussian: Learning Generalizable Gaussian Human with Integrated Human Features

  • 基于通用高斯点云融合姿态回归与特征点渲染
  • 25帧/秒实现实时重建,支持骨骼、关键点等多维特征输出
  • 适合需快速生成带生物力学信息的人体数字孪生场景

最近的辐射场渲染进展在3D场景表示上取得显著成果,其中基于高斯点云的技术因高质量和高效率成为新标准。该技术被广泛应用于3D人体建模,但现有方法或依赖参数化人体模型作为额外信息,或缺乏人体生物力学结构(如骨骼、关键点)的支持,难以满足多样化应用需求。本文提出HFGaussian方法,可从稀疏输入图像中实时(25 FPS)重建新视角下的3D人体,并同步估计3D骨骼、3D关键点及稠密姿态。该方法利用可泛化的高斯点云表示人体及其关联特征,结合姿态回归网络与特征点云渲染技术,显著提升现有3D人体建模与姿态估计方法的表现。我们在多个最新人体高斯点云与姿态估计基准上进行了全面评估,验证了HFGaussian在实时性与性能上的领先优势。

原文摘要 · Abstract (English)

Recent advancements in radiance field rendering show promising results in 3D scene representation, where Gaussian splatting-based techniques emerge as state-of-the-art due to their quality and efficiency. Gaussian splatting is widely used for various applications, including 3D human representation. However, previous 3D Gaussian splatting methods either use parametric body models as additional information or fail to provide any underlying structure, like human biomechanical features, which are essential for different applications. In this paper, we present a novel approach called HFGaussian that can estimate novel views and human features, such as the 3D skeleton, 3D key points, and dense pose, from sparse input images in real time at 25 FPS. The proposed method leverages generalizable Gaussian splatting technique to represent the human subject and its associated features, enabling efficient and generalizable reconstruction. By incorporating a pose regression network and the feature splatting technique with Gaussian splatting, HFGaussian demonstrates improved capabilities over existing 3D human methods, showcasing the potential of 3D human representations with integrated biomechanics. We thoroughly evaluate our HFGaussian method against the latest state-of-the-art techniques in human Gaussian splatting and pose estimation, demonstrating its real-time, state-of-the-art performance.

3D人体重建高斯点云姿态估计实时渲染

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。