提出双向增强的高斯点云方法,实现光照变化下的高质量渲染与精确重建。
MGSR: 2D/3D Mutual-boosted Gaussian Splatting for High-fidelity Surface Reconstruction under Various Light Conditions
- 2D/3D双分支互馈优化,几何与光照信息双向增强。
- 在真实场景和合成数据上均达到优于现有方法的渲染与重建精度。
- 适合需要高保真3D重建与复杂光照模拟的应用场景。
新视角合成(NVS)与表面重建(SR)是3D高斯点云(3D-GS)中的关键任务。尽管近期取得进展,这些任务常被独立处理:基于GS的渲染方法在多样光照下表现不佳,而基于GS的重建方法常牺牲渲染质量。这引发核心问题:渲染与重建是否必须权衡?为此,我们提出MGSR——一种用于表面重建的2D/3D双向增强高斯点云方法,同时提升渲染质量和3D重建精度。MGSR包含两个分支:2D-GS分支擅长表面重建,为3D-GS分支提供精确几何信息;3D-GS分支利用该几何信息,通过几何引导的光照分解模块分离反射与透射成分,实现在多种光照条件下的逼真渲染。再以透射成分作为监督信号,2D-GS分支也实现了高保真表面重建。优化过程中,两个分支交替进行,相互监督。此前,各分支先独立完成预热阶段,并采用早停策略降低计算成本。我们在多样化的合成与真实世界数据集(对象级与场景级)上评估了MGSR,验证其在渲染与重建方面的强大性能。代码已开源:https://github.com/TsingyuanChou/MGSR。
原文摘要 · Abstract (English)
Novel view synthesis (NVS) and surface reconstruction (SR) are essential tasks in 3D Gaussian Splatting (3D-GS). Despite recent progress, these tasks are often addressed independently, with GS-based rendering methods struggling under diverse light conditions and failing to produce accurate surfaces, while GS-based reconstruction methods frequently compromise rendering quality. This raises a central question: must rendering and reconstruction always involve a trade-off? To address this, we propose MGSR, a 2D/3D Mutual-boosted Gaussian splatting for Surface Reconstruction that enhances both rendering quality and 3D reconstruction accuracy. MGSR introduces two branches--one based on 2D-GS and the other on 3D-GS. The 2D-GS branch excels in surface reconstruction, providing precise geometry information to the 3D-GS branch. Leveraging this geometry, the 3D-GS branch employs a geometry-guided illumination decomposition module that captures reflected and transmitted components, enabling realistic rendering under varied light conditions. Using the transmitted component as supervision, the 2D-GS branch also achieves high-fidelity surface reconstruction. Throughout the optimization process, the 2D-GS and 3D-GS branches undergo alternating optimization, providing mutual supervision. Prior to this, each branch completes an independent warm-up phase, with an early stopping strategy implemented to reduce computational costs. We evaluate MGSR on a diverse set of synthetic and real-world datasets, at both object and scene levels, demonstrating strong performance in rendering and surface reconstruction. Code is available at https://github.com/TsingyuanChou/MGSR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。