分离几何与外观预测，实现高效无相机参数的3D高斯溅射重建
Stereo-GS: Multi-View Stereo Vision Model for Generalizable 3D Gaussian Splatting Reconstruction
- 通过双目视觉提取局部特征，用全局注意力融合多视角信息
- 解耦几何与外观生成，使重建速度更快且资源消耗更低
- 无需相机参数即可完成重建，适合实际场景快速建模
通用3D高斯溅射重建在图像到3D内容生成方面表现优异，但训练需大量计算资源和数据，难以从零开始。现有方法常将3D高斯的几何与外观预测耦合，依赖数据驱动先验，导致回归速度慢。为此，我们提出一种解耦框架，通过双目视觉主干网络提取局部图像对特征，并利用全局注意力块进行融合。专用的点预测头与高斯预测头分别生成多视角点图（geometry）与高斯特征图（appearance），合并为GS图表示3DGS对象。再通过优化网络提升GS图质量，实现高质量重建。该方法不依赖相机参数，可实现无姿态3D重建，增强鲁棒性与实用性。在降低资源需求的同时保持高质量输出，为真实世界3D内容生成提供高效可扩展的解决方案。
原文摘要 · Abstract (English)
Generalizable 3D Gaussian Splatting reconstruction showcases advanced Image-to-3D content creation but requires substantial computational resources and large datasets, posing challenges to training models from scratch. Current methods usually entangle the prediction of 3D Gaussian geometry and appearance, which rely heavily on data-driven priors and result in slow regression speeds. To address this, we propose \method, a disentangled framework for efficient 3D Gaussian prediction. Our method extracts features from local image pairs using a stereo vision backbone and fuses them via global attention blocks. Dedicated point and Gaussian prediction heads generate multi-view point-maps for geometry and Gaussian features for appearance, combined as GS-maps to represent the 3DGS object. A refinement network enhances these GS-maps for high-quality reconstruction. Unlike existing methods that depend on camera parameters, our approach achieves pose-free 3D reconstruction, improving robustness and practicality. By reducing resource demands while maintaining high-quality outputs, \method provides an efficient, scalable solution for real-world 3D content generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。