构建真实雨滴干扰下的3D高斯溅射评估基准,提升实际场景重建能力。
RaindropGS: A Benchmark for 3D Gaussian Splatting under Raindrop Conditions
- 基于真实雨滴图像,构建从模糊到清晰的完整3DGS评估流程。
- 发现焦距位置与相机位姿估计误差显著影响重建质量。
- 适合研究视觉导航、自动驾驶中恶劣天气鲁棒性方法的团队。
在雨滴污染镜头条件下,3D高斯溅射(3DGS)面临严重遮挡与光学畸变,导致重建质量大幅下降。现有基准多使用已知相机位姿的合成雨滴图像进行评估(受限图像),假设理想条件。然而真实场景中雨滴会干扰相机位姿估计与点云初始化,且合成与真实雨滴间存在显著领域差距,影响泛化性。为此,我们提出RaindropGS,一个涵盖从非约束雨滴干扰图像到清晰3DGS重建的全流程评估基准。该基准包含三部分:数据准备、数据处理及雨滴感知的3DGS评估,涉及雨滴干扰类型、相机位姿估计与点云初始化、单图去雨对比、3D高斯训练对比。首先,收集真实世界雨滴重建数据集,每个场景包含三组对齐图像:雨滴聚焦、背景聚焦与无雨真值,支持不同聚焦条件下的重建质量评估。通过全面实验与分析,揭示现有3DGS方法在非约束雨滴图像上的性能瓶颈,以及各环节的影响差异:焦距位置对3DGS重建的影响,以及位姿与点云初始化不准确带来的干扰。这些洞察为开发更鲁棒的雨滴环境下3DGS方法提供了明确方向。
原文摘要 · Abstract (English)
3D Gaussian Splatting (3DGS) under raindrop conditions suffers from severe occlusions and optical distortions caused by raindrop contamination on the camera lens, substantially degrading reconstruction quality. Existing benchmarks typically evaluate 3DGS using synthetic raindrop images with known camera poses (constrained images), assuming ideal conditions. However, in real-world scenarios, raindrops often interfere with accurate camera pose estimation and point cloud initialization. Moreover, a significant domain gap between synthetic and real raindrops further impairs generalization. To tackle these issues, we introduce RaindropGS, a comprehensive benchmark designed to evaluate the full 3DGS pipeline-from unconstrained, raindrop-corrupted images to clear 3DGS reconstructions. Specifically, the whole benchmark pipeline consists of three parts: data preparation, data processing, and raindrop-aware 3DGS evaluation, including types of raindrop interference, camera pose estimation and point cloud initialization, single image rain removal comparison, and 3D Gaussian training comparison. First, we collect a real-world raindrop reconstruction dataset, in which each scene contains three aligned image sets: raindrop-focused, background-focused, and rain-free ground truth, enabling a comprehensive evaluation of reconstruction quality under different focus conditions. Through comprehensive experiments and analyses, we reveal critical insights into the performance limitations of existing 3DGS methods on unconstrained raindrop images and the varying impact of different pipeline components: the impact of camera focus position on 3DGS reconstruction performance, and the interference caused by inaccurate pose and point cloud initialization on reconstruction. These insights establish clear directions for developing more robust 3DGS methods under raindrop conditions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。