无需真实相机位姿，从稀疏视角实现高效3D高斯点云重建
SPFSplatV2: Efficient Self-Supervised Pose-Free 3D Gaussian Splatting from Sparse Views
- 共享主干网络联合预测3D高斯点与相机位姿
- 在无位姿监督下仍达顶尖新视角合成效果
- 适合大规模多样化数据集的可扩展重建
我们提出SPFSplatV2，一种高效的前馈框架，仅需稀疏多视角图像即可完成3D高斯点云重建，训练与推理均无需真实相机位姿。该框架采用共享特征提取主干，在规范空间中联合预测3D高斯原语与相机位姿。为实现高效精确的位姿估计，引入掩码注意力机制进行目标视图位姿预测，并设计重投影损失以强制高斯点与像素对齐，提供更强几何约束。我们进一步验证了该训练框架与不同重建架构的兼容性，衍生出两种模型变体。值得注意的是，尽管无位姿监督，本方法在域内与域外新视角合成任务中均达到当前最优性能，即使在极端视角变化和有限图像重叠条件下依然表现优异。其相对位姿估计精度也超越许多依赖几何监督的方法。通过消除对真实位姿的依赖，本方法显著提升了利用更大更多样数据集的可扩展性。代码与预训练模型将公开于项目主页：https://ranrhuang.github.io/spfsplatv2/。
原文摘要 · Abstract (English)
We introduce SPFSplatV2, an efficient feed-forward framework for 3D Gaussian splatting from sparse multi-view images, requiring no ground-truth poses during training or inference. The framework employs a shared feature extraction backbone to jointly predict 3D Gaussian primitives and camera poses in a canonical space from unposed inputs. To enable efficient and accurate pose estimation, we introduce a masked attention mechanism for target-view pose prediction and a reprojection loss that enforces pixel-aligned Gaussian primitives, providing stronger geometric constraints. We further demonstrate the compatibility of our training framework with different reconstruction architectures, resulting in two model variants. Remarkably, despite the absence of pose supervision, our method achieves state-of-the-art performance in both in-domain and out-of-domain novel view synthesis, even under extreme viewpoint changes and limited image overlap. It also surpasses many methods that rely on geometric supervision in relative pose estimation. By eliminating dependence on ground-truth poses, our method offers the scalability to leverage larger and more diverse datasets. Code and pretrained models will be available on our project page: https://ranrhuang.github.io/spfsplatv2/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。