针对恶劣天气下3D重建模糊问题,提出分离雨雪颗粒与镜头遮挡的重建方法。
WeatherGS: 3D Scene Reconstruction in Adverse Weather Conditions via Gaussian Splatting
- 将天气干扰分为空中雨雪颗粒和镜头遮挡两类,分别处理
- 通过滤波与检测去除90%以上颗粒与遮挡,提升重建清晰度
- 适合自动驾驶、机器人视觉等户外复杂环境应用
3D Gaussian Splatting(3DGS)在3D场景重建中表现优异,但在恶劣天气下仍受限,因会将天气造成的伪影当作真实场景重建,导致画面模糊。为解决此问题,本文提出WeatherGS,一种基于3DGS的多视角图像重建框架,可在不同天气条件下恢复清晰场景。具体地,将多天气伪影分为具有高密度特性的空中颗粒(如雨滴、雪花)和镜头表面遮挡,并提出稀疏化预处理策略:先用大气效应滤波器(AEF)去除密集颗粒,再用镜头效应检测器(LED)提取稀疏遮挡掩码。随后,利用处理后的图像和生成的掩码训练3D高斯,并在高斯点渲染时排除被遮挡区域,实现对底层清晰场景的准确重建。我们构建了一个多样且具有挑战性的基准测试集,用于评估复杂天气下的3D重建性能。大量实验表明,WeatherGS在多种天气场景下均能持续生成高质量、清晰的场景,优于现有最先进方法。
原文摘要 · Abstract (English)
3D Gaussian Splatting (3DGS) has gained significant attention for 3D scene reconstruction, but still suffers from complex outdoor environments, especially under adverse weather. This is because 3DGS treats the artifacts caused by adverse weather as part of the scene and will directly reconstruct them, largely reducing the clarity of the reconstructed scene. To address this challenge, we propose WeatherGS, a 3DGS-based framework for reconstructing clear scenes from multi-view images under different weather conditions. Specifically, we explicitly categorize the multi-weather artifacts into the dense particles and lens occlusions that have very different characters, in which the former are caused by snowflakes and raindrops in the air, and the latter are raised by the precipitation on the camera lens. In light of this, we propose a dense-to-sparse preprocess strategy, which sequentially removes the dense particles by an Atmospheric Effect Filter (AEF) and then extracts the relatively sparse occlusion masks with a Lens Effect Detector (LED). Finally, we train a set of 3D Gaussians by the processed images and generated masks for excluding occluded areas, and accurately recover the underlying clear scene by Gaussian splatting. We conduct a diverse and challenging benchmark to facilitate the evaluation of 3D reconstruction under complex weather scenarios. Extensive experiments on this benchmark demonstrate that our WeatherGS consistently produces high-quality, clean scenes across various weather scenarios, outperforming existing state-of-the-art methods. See project page:https://jumponthemoon.github.io/weather-gs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。