通过迭代优化深度概率,提升3D高斯点云重建精度与泛化能力。
IDESplat: Iterative Depth Probability Estimation for Generalizable 3D Gaussian Splatting
- 采用多级几何扭曲与乘性注意力融合,逐步增强深度概率估计。
- 在RealEstate10K上比DepthSplat高0.33 dB PSNR,参数仅用10.7%。
- 适合追求高效高精度3D重建的科研与工业应用。
通用3D高斯点云重建旨在直接通过前馈网络预测高斯参数。其中高斯中心(即位置)难以准确预测,通常先估计深度再反投影获得。现有方法仅依赖单次扭曲操作进行深度估计,难以充分挖掘跨视角几何线索,导致深度图不稳定且粗糙。为此,本文提出IDESplat,通过迭代应用扭曲操作来提升深度概率估计,实现更精准的高斯中心预测。首先,设计深度概率增强单元(DPBU),以乘性方式融合级联扭曲生成的对极注意力图,消除单次扭曲的不稳定性;其次,通过堆叠多个DPBU构建迭代深度估计流程,逐步筛选高概率深度候选。随着迭代进行,深度图不断精炼,最终实现精确的高斯中心预测。在RealEstate10K、ACID和DL3DV数据集上实验表明,IDESplat在保持实时效率的同时,取得领先重建质量。在RE10K上,相比DepthSplat提升0.33 dB PSNR,参数量仅10.7%,内存占用70%;在跨数据集测试中,于DTU上较DepthSplat提升2.95 dB PSNR,展现强大泛化能力。
原文摘要 · Abstract (English)
Generalizable 3D Gaussian Splatting aims to directly predict Gaussian parameters using a feed-forward network for scene reconstruction. Among these parameters, Gaussian means are particularly difficult to predict, so depth is usually estimated first and then unprojected to obtain the Gaussian sphere centers. Existing methods typically rely solely on a single warp to estimate depth probability, which hinders their ability to fully leverage cross-view geometric cues, resulting in unstable and coarse depth maps. To address this limitation, we propose IDESplat, which iteratively applies warp operations to boost depth probability estimation for accurate Gaussian mean prediction. First, to eliminate the inherent instability of a single warp, we introduce a Depth Probability Boosting Unit (DPBU) that integrates epipolar attention maps produced by cascading warp operations in a multiplicative manner. Next, we construct an iterative depth estimation process by stacking multiple DPBUs, progressively identifying potential depth candidates with high likelihood. As IDESplat iteratively boosts depth probability estimates and updates the depth candidates, the depth map is gradually refined, resulting in accurate Gaussian means. We conduct experiments on RealEstate10K, ACID, and DL3DV. IDESplat achieves outstanding reconstruction quality and state-of-the-art performance with real-time efficiency. On RE10K, it outperforms DepthSplat by 0.33 dB in PSNR, using only 10.7% of the parameters and 70% of the memory. Additionally, our IDESplat improves PSNR by 2.95 dB over DepthSplat on the DTU dataset in cross-dataset experiments, demonstrating its strong generalization ability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。