arXiv:2505.15294cs.CVcs.GR2025-05

R3GS提升无约束图像集的重建与重定位精度,兼顾动态物体和天空区域处理。

R3GS: Gaussian Splatting for Robust Reconstruction and Relocalization in Unconstrained Image Collections

  • 混合表示:用CNN全局特征+多分辨率哈希编码局部特征
  • 轻量级人体检测网络生成可见性图,有效抑制动态物体干扰
  • 天空建模引入深度先验,减少无限远天空的漂浮伪影

我们提出R3GS,一种针对无约束数据集的鲁棒重建与重定位框架。训练时采用混合表示:每个锚点结合卷积神经网络(CNN)的全局特征与多分辨率哈希网格[2]编码的局部特征。随后,多个浅层多层感知机(MLPs)预测每个高斯体的颜色、不透明度和协方差。为缓解瞬态物体对重建的负面影响,我们微调了一个轻量级人体检测网络,生成可见性图,能高效泛化至其他瞬态物体(如海报、旗帜、车辆),且几乎无需进一步适配。此外,针对户外场景中的天空区域,我们提出一种有效的天空处理技术,引入深度先验作为约束,使无穷远天空被表示在大半径天空球面上,显著减少因天空重建误差导致的漂浮伪影。同时,我们引入一种新型重定位方法,在光照变化下仍能稳定估计给定图像在重建的3DGS场景中的相机位姿。实验表明,R3GS显著提升渲染保真度,改善训练与渲染效率,并降低存储开销。在真实世界数据集上优于基线方法,达到当前最优性能。代码将在论文录用后开源。

原文摘要 · Abstract (English)

We propose R3GS, a robust reconstruction and relocalization framework tailored for unconstrained datasets. Our method uses a hybrid representation during training. Each anchor combines a global feature from a convolutional neural network (CNN) with a local feature encoded by the multiresolution hash grids [2]. Subsequently, several shallow multi-layer perceptrons (MLPs) predict the attributes of each Gaussians, including color, opacity, and covariance. To mitigate the adverse effects of transient objects on the reconstruction process, we ffne-tune a lightweight human detection network. Once ffne-tuned, this network generates a visibility map that efffciently generalizes to other transient objects (such as posters, banners, and cars) with minimal need for further adaptation. Additionally, to address the challenges posed by sky regions in outdoor scenes, we propose an effective sky-handling technique that incorporates a depth prior as a constraint. This allows the inffnitely distant sky to be represented on the surface of a large-radius sky sphere, signiffcantly reducing ffoaters caused by errors in sky reconstruction. Furthermore, we introduce a novel relocalization method that remains robust to changes in lighting conditions while estimating the camera pose of a given image within the reconstructed 3DGS scene. As a result, R3GS significantly enhances rendering ffdelity, improves both training and rendering efffciency, and reduces storage requirements. Our method achieves state-of-the-art performance compared to baseline methods on in-the-wild datasets. The code will be made open-source following the acceptance of the paper.

三维重建高斯溅射重定位天空建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。