用语义对齐提升稀疏视角3D重建质量，让少图也能生成逼真图像。
SPARS3R: Semantic Prior Alignment and Regularization for Sparse 3D Reconstruction
- 先全局对齐稠密点云与稀疏点云，再局部修复语义不一致区域。
- 在仅用少量视图下实现逼真渲染，性能显著优于现有方法。
- 适合做稀疏视角3D重建、新视角合成的科研与工程人员。
基于高斯溅射的新视角合成近期已实现逼真渲染，但在稀疏视图场景中受限于初始化稀疏和浮点噪声过拟合问题。尽管深度估计与对齐技术可从少数视图生成稠密点云，但姿态精度仍不理想。本文提出SPARS3R，融合运动恢复结构（SfM）的精准姿态估计与深度估计的稠密点云优势。首先通过全局融合对齐，基于三角化对应关系将先验稠密点云映射至基于SfM的稀疏点云，并利用RANSAC区分内点与外点。随后进行语义异常对齐步骤，提取外点周围的语义一致区域并执行局部对齐。结合评估流程改进，SPARS3R在少量图像条件下实现了逼真渲染，显著超越现有方法。
原文摘要 · Abstract (English)
Recent efforts in Gaussian-Splat-based Novel View Synthesis can achieve photorealistic rendering; however, such capability is limited in sparse-view scenarios due to sparse initialization and over-fitting floaters. Recent progress in depth estimation and alignment can provide dense point cloud with few views; however, the resulting pose accuracy is suboptimal. In this work, we present SPARS3R, which combines the advantages of accurate pose estimation from Structure-from-Motion and dense point cloud from depth estimation. To this end, SPARS3R first performs a Global Fusion Alignment process that maps a prior dense point cloud to a sparse point cloud from Structure-from-Motion based on triangulated correspondences. RANSAC is applied during this process to distinguish inliers and outliers. SPARS3R then performs a second, Semantic Outlier Alignment step, which extracts semantically coherent regions around the outliers and performs local alignment in these regions. Along with several improvements in the evaluation process, we demonstrate that SPARS3R can achieve photorealistic rendering with sparse images and significantly outperforms existing approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。