从少量雨天视角重建清晰3D场景,支持自动驾驶等空间智能应用。
DerainSplat: Feed-Forward Clean 3D Gaussian Splatting from Sparse Rainy Views

- 构建多阶段合成数据集,模拟真实雨天光照与雨滴效果。
- 引入天气网络预测气象因子,实现跨视角匹配与图像修复。
- 端到端训练,可在真实雨天数据上泛化,适合车载感知系统。
尽管图像去雨技术已取得显著进展,但现有方法主要集中于2D图像修复。随着具身智能与自动驾驶等空间智能应用的发展,如何以前馈方式从稀疏雨天视角重建清晰3D场景变得愈发重要。现有前馈3D高斯溅射(3DGS)方法通常假设输入为清洁图像,在雨天条件下性能严重退化。为此,我们提出【DerainSplat】——一种从前馈角度重建干净3D场景的框架,仅需少数雨天视角即可完成。为支持该任务,我们通过四阶段合成流程构建大规模多视角去雨数据集,依次建模阴天光照、依赖深度的雾效、雨痕及镜头雨滴,生成具有优势天气因素的真实雨景。我们引入天气网络,从雨天上下文预测气象因素并生成两张辅助图:场景支持用于调节跨视角代价体积匹配,辐射支持驱动深度对齐的外观融合以填补受损像素。推导出的几何证据进一步衰减高斯透明度,抑制伪结构。通过雨天循环一致性机制,利用预测因素重新渲染干净视图,并与原始雨天输入对齐。大量实验表明,DerainSplat在RealEstate10K、ACID、Mip-NeRF360及真实雨天场景等多个数据集上均优于现有方法,具备强跨数据集泛化能力。
原文摘要 · Abstract (English)
Although image deraining has advanced substantially, existing methods mainly focus on 2D image restoration. As spatial intelligence applications such as embodied AI and autonomous driving continue to emerge, reconstructing clean 3D scenes from sparse rainy views in a feed-forward manner becomes increasingly important. Existing feed-forward 3D Gaussian Splatting (3DGS) methods often assume clean inputs and collapse under rainy conditions. To this end, we present \textbf{\textit{DerainSplat}}, a feed-forward framework that reconstructs clean 3D scenes from only a few rainy views. To support this task, we build a large-scale multi-view derain dataset through a four-stage synthesis pipeline that sequentially models overcast illumination, depth-dependent haze, rain streaks, and lens raindrops, producing privileged weather factors. We introduce a weather net that predicts the weather factors from rainy context and yields two support maps. Scene support modulates cross-view cost-volume matching, while radiance support drives depth-aligned appearance fusion to fill corrupted pixels. The derived geometry evidence further attenuates Gaussian opacity to reduce spurious structures. A rainy cycle consistency re-renders clean views using the predicted factors and aligns them with rainy inputs. Extensive experiments show that \textbf{\textit{DerainSplat}} outperforms existing methods on various datasets, including RealEstate10K, ACID, Mip-NeRF360, and real-world rainy scenes, with strong cross-dataset generalization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。