arXiv:2501.14319cs.CVcs.RO2025-01ICLR被引 10

提出新基准与方法,让3D重建在真实噪声环境下更稳定可靠。

Scalable Benchmarking and Robust Learning for Noise-Free Ego-Motion and 3D Reconstruction from Noisy Video

  • 构建可扩展的噪声数据生成管道,模拟复杂真实场景扰动。
  • 在噪声环境下,模型性能显著优于现有方法,尤其动态光照下。
  • 适合做鲁棒视觉感知、自动驾驶等真实场景应用的研究者。

我们旨在通过解决现有模型依赖无噪声数据这一关键局限,重新定义鲁棒的自运动估计与逼真3D重建。当前方法在理想条件下表现良好,但面对真实世界中的动态运动、传感器缺陷和同步误差时性能急剧下降,凸显了面向真实噪声环境的迫切需求。为此,我们应对三大挑战:可扩展的数据生成、全面的基准测试与模型鲁棒性增强。首先,提出一种可扩展的噪声数据合成流程,生成涵盖复杂运动、传感器缺陷与同步误差的多样化数据集。其次,基于该流程构建Robust-Ego3D基准,系统暴露噪声导致的性能退化,揭示现有学习方法在自运动精度与3D重建质量上的不足。第三,提出对应引导的高斯点云渲染(CorrGS),一种测试时自适应方法,通过将观测图像与干净3D地图渲染的RGB-D帧对齐,逐步优化内部干净3D表示,提升几何对齐与外观复原。在合成与真实数据上的大量实验表明,CorrGS在快速运动与动态光照等极端场景中持续优于现有最先进方法。

原文摘要 · Abstract (English)

We aim to redefine robust ego-motion estimation and photorealistic 3D reconstruction by addressing a critical limitation: the reliance on noise-free data in existing models. While such sanitized conditions simplify evaluation, they fail to capture the unpredictable, noisy complexities of real-world environments. Dynamic motion, sensor imperfections, and synchronization perturbations lead to sharp performance declines when these models are deployed in practice, revealing an urgent need for frameworks that embrace and excel under real-world noise. To bridge this gap, we tackle three core challenges: scalable data generation, comprehensive benchmarking, and model robustness enhancement. First, we introduce a scalable noisy data synthesis pipeline that generates diverse datasets simulating complex motion, sensor imperfections, and synchronization errors. Second, we leverage this pipeline to create Robust-Ego3D, a benchmark rigorously designed to expose noise-induced performance degradation, highlighting the limitations of current learning-based methods in ego-motion accuracy and 3D reconstruction quality. Third, we propose Correspondence-guided Gaussian Splatting (CorrGS), a novel test-time adaptation method that progressively refines an internal clean 3D representation by aligning noisy observations with rendered RGB-D frames from clean 3D map, enhancing geometric alignment and appearance restoration through visual correspondence. Extensive experiments on synthetic and real-world data demonstrate that CorrGS consistently outperforms prior state-of-the-art methods, particularly in scenarios involving rapid motion and dynamic illumination.

3D重建自运动估计噪声鲁棒高斯点云

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。