无需真实相机参数，用稀疏多视角图重建高质量3D场景。
SmileSplat: Generalizable Gaussian Splats for Unconstrained Sparse Images
- 通过多头高斯回归预测像素对齐的高斯点元，提升多视角一致性。
- 联合优化高斯点与相机参数，实现端到端的高质量新视角合成。
- 适用于无约束稀疏图像，适合实际应用中的3D重建任务。
稀疏多视角图像可通过通用高斯点元方法学习显式辐射场，当无需输入真实相机参数时，在真实场景中具有更广泛的应用前景。本文提出一种新型通用高斯点元方法 SmileSplat，仅需非约束的稀疏多视角图像即可重建像素对齐的高斯点元，适用于多样场景。首先，基于多头高斯回归解码器预测高斯点元，自由度更低且多视角一致性更强；同时利用高质量法向先验增强点元法向。其次，通过提出的束调整高斯点元模块，联合优化高斯点与相机参数（内外参），获得高质量的高斯辐射场，用于新视角合成任务。在多个公开数据集上进行了新视角渲染与深度图预测的大量实验，结果表明该方法在多种3D视觉任务中达到当前最优性能。
原文摘要 · Abstract (English)
Sparse Multi-view Images can be Learned to predict explicit radiance fields via Generalizable Gaussian Splatting approaches, which can achieve wider application prospects in real-life when ground-truth camera parameters are not required as inputs. In this paper, a novel generalizable Gaussian Splatting method, SmileSplat, is proposed to reconstruct pixel-aligned Gaussian surfels for diverse scenarios only requiring unconstrained sparse multi-view images. First, Gaussian surfels are predicted based on the multi-head Gaussian regression decoder, which can are represented with less degree-of-freedom but have better multi-view consistency. Furthermore, the normal vectors of Gaussian surfel are enhanced based on high-quality of normal priors. Second, the Gaussians and camera parameters (both extrinsic and intrinsic) are optimized to obtain high-quality Gaussian radiance fields for novel view synthesis tasks based on the proposed Bundle-Adjusting Gaussian Splatting module. Extensive experiments on novel view rendering and depth map prediction tasks are conducted on public datasets, demonstrating that the proposed method achieves state-of-the-art performance in various 3D vision tasks. More information can be found on our project page (https://yanyan-li.github.io/project/gs/smilesplat)
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。