从稀疏多视角图生成高保真人像新视图，无需逐人优化。
RoGSplat: Learning Robust Generalizable Human Gaussian Splatting from Sparse Multi-View Images
- 用SMPL顶点生成密集3D先验点，回归人体高斯参数。
- 在少重叠视角下仍能重建复杂人体几何，性能超越当前最佳。
- 适合需要快速泛化到新人物的三维人像重建场景。
本文提出RoGSplat，一种从稀疏多视角图像中合成高质量未见视角人像的新方法，且无需对每个个体进行繁琐的优化。针对以往方法在视角稀疏、重叠少时重建效果差的问题，本方法通过将SMPL顶点提升为密集可靠的3D先验点，准确表示人体几何，并基于这些点回归人体高斯参数。为应对SMPL模型与图像间的可能错位，我们结合像素级和体素级特征预测图像对齐的3D先验点，进而回归粗粒度高斯；为进一步捕捉高频细节，我们从粗粒度3D高斯渲染深度图，辅助回归细粒度像素级高斯。在多个基准数据集上的实验表明，该方法在新视图合成与跨数据集泛化方面均优于现有最先进方法。代码已开源：https://github.com/iSEE-Laboratory/RoGSplat。
原文摘要 · Abstract (English)
This paper presents RoGSplat, a novel approach for synthesizing high-fidelity novel views of unseen human from sparse multi-view images, while requiring no cumbersome per-subject optimization. Unlike previous methods that typically struggle with sparse views with few overlappings and are less effective in reconstructing complex human geometry, the proposed method enables robust reconstruction in such challenging conditions. Our key idea is to lift SMPL vertices to dense and reliable 3D prior points representing accurate human body geometry, and then regress human Gaussian parameters based on the points. To account for possible misalignment between SMPL model and images, we propose to predict image-aligned 3D prior points by leveraging both pixel-level features and voxel-level features, from which we regress the coarse Gaussians. To enhance the ability to capture high-frequency details, we further render depth maps from the coarse 3D Gaussians to help regress fine-grained pixel-wise Gaussians. Experiments on several benchmark datasets demonstrate that our method outperforms state-of-the-art methods in novel view synthesis and cross-dataset generalization. Our code is available at https://github.com/iSEE-Laboratory/RoGSplat.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。