一个模型搞定任意图像的3D高斯点云重建,无需校准和位姿。
YoNoSplat: You Only Need One Model for Feedforward 3D Gaussian Splatting

- 单模型统一处理有无位姿、是否校准的输入图像。
- 100张图重建仅需2.69秒,性能超越现有方法。
- 适合快速3D重建场景,尤其适合无标定数据的用户。
从无序图像集合中实现快速灵活的3D场景重建仍是重大挑战。我们提出YoNoSplat,一种前馈模型,能从任意数量图像中重建高质量3D高斯点云表示。该模型高度灵活,可处理带位姿与无位姿、已校准与未校准的输入。YoNoSplat为每视图预测局部高斯分布与相机位姿,并通过预测或提供的位姿聚合为全局表示。为克服联合学习3D高斯与相机参数的固有难题,我们提出一种新颖的混合训练策略:初期使用真值位姿聚合局部高斯,逐步过渡到预测与真值混合,有效避免训练不稳定与暴露偏差。我们还通过成对相机距离归一化解决尺度模糊问题,并将相机内参嵌入网络。此外,模型可预测内参,支持未校准输入。在标准基准上,无论是否有位姿,其表现均达当前最优。在NVIDIA GH200 GPU上,100张图(280x518分辨率)重建仅耗时2.69秒。
原文摘要 · Abstract (English)
Fast and flexible 3D scene reconstruction from unstructured image collections remains a significant challenge. We present YoNoSplat, a feedforward model that reconstructs high-quality 3D Gaussian Splatting representations from an arbitrary number of images. Our model is highly versatile, operating effectively with both posed and unposed, calibrated and uncalibrated inputs. YoNoSplat predicts local Gaussians and camera poses for each view, which are aggregated into a global representation using either predicted or provided poses. To overcome the inherent difficulty of jointly learning 3D Gaussians and camera parameters, we introduce a novel mixing training strategy. This approach mitigates the entanglement between the two tasks by initially using ground-truth poses to aggregate local Gaussians and gradually transitioning to a mix of predicted and ground-truth poses, which prevents both training instability and exposure bias. We further resolve the scale ambiguity problem by a novel pairwise camera-distance normalization scheme and by embedding camera intrinsics into the network. Moreover, YoNoSplat also predicts intrinsic parameters, making it feasible for uncalibrated inputs. YoNoSplat demonstrates exceptional efficiency, reconstructing a scene from 100 views (at 280x518 resolution) in just 2.69 seconds on an NVIDIA GH200 GPU. It achieves state-of-the-art performance on standard benchmarks in both pose-free and pose-dependent settings. Our project page is at https://botaoye.github.io/yonosplat/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。