用最少输入实现高质量3D高斯溅射重建
S2D: Sparse to Dense Lifting for 3D Reconstruction with Minimal Inputs
- 通过扩散模型将稀疏点云直接升维为稠密表示
- 在不同稀疏度下均达到顶尖的新视角生成一致性
- 适合低采集成本的3D重建场景
显式3D表示已成为3D模拟与理解的核心手段。然而，当前主流的点云和3D高斯溅射（3DGS）在稀疏输入下分别面临渲染不真实与显著退化的问题。本文提出稀疏到稠密提升（S2D），一种新管道，可在最小输入条件下实现高质量3DGS重建。S2D包含两方面：首先设计高效单步扩散模型，修复稀疏点云的图像伪影；其次引入随机采样丢弃与加权梯度策略，实现从稀疏视角到稠密新视角的鲁棒场景重建。大量实验表明，S2D在不同输入稀疏度下均取得最佳新视角生成一致性与顶级稀疏视图重建质量。相比现有方法，S2D以最少拍摄数据重建稳定场景，满足3DGS应用的极低输入需求。
原文摘要 · Abstract (English)
Explicit 3D representations have already become an essential medium for 3D simulation and understanding. However, the most commonly used point cloud and 3D Gaussian Splatting (3DGS) each suffer from non-photorealistic rendering and significant degradation under sparse inputs. In this paper, we introduce Sparse to Dense lifting (S2D), a novel pipeline that bridges the two representations and achieves high-quality 3DGS reconstruction with minimal inputs. Specifically, the S2D lifting is two-fold. We first present an efficient one-step diffusion model that lifts sparse point cloud for high-fidelity image artifact fixing. Meanwhile, to reconstruct 3D consistent scenes, we also design a corresponding reconstruction strategy with random sample drop and weighted gradient for robust model fitting from sparse input views to dense novel views. Extensive experiments show that S2D achieves the best consistency in generating novel view guidance and first-tier sparse view reconstruction quality under different input sparsity. By reconstructing stable scenes with the least possible captures among existing methods, S2D enables minimal input requirements for 3DGS applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。