arXiv:2602.17124cs.CVcs.AI2026-02

用雷达数据辅助3D高斯点云初始化,提升恶劣条件下的渲染鲁棒性

3D Scene Rendering with Multimodal Gaussian Splatting

  • 融合雷达与视觉信息,用稀疏雷达深度图生成高质量3D点云
  • 在低光、遮挡等条件下仍能实现高保真3D场景重建
  • 适合自动驾驶、机器人等需全天候感知的场景

3D场景重建与渲染是计算机视觉的核心任务,广泛应用于工业监控、机器人和自动驾驶等领域。近年来,基于3D高斯泼溅(GS)及其变体的方法在保持高计算与内存效率的同时,实现了出色的渲染质量。然而,传统基于视觉的GS流程通常依赖足够多的相机视角来初始化高斯原语并训练其参数,这在初始化阶段带来额外开销,且在视觉线索不可靠的条件下(如恶劣天气、低光照或部分遮挡)表现不佳。为应对这些挑战,并受射频(RF)信号对天气、光照和遮挡具有强鲁棒性的启发,我们提出一种多模态框架,将车载雷达等射频传感与基于GS的渲染相结合,作为比纯视觉方法更高效、更鲁棒的替代方案。该方法仅需稀疏的基于雷达的深度测量即可实现高效的深度预测,生成高质量3D点云以初始化各类GS架构中的高斯函数。数值实验表明,合理融入射频感知可显著提升GS流水线的性能,实现由射频引导的结构精度驱动的高保真3D场景渲染。

原文摘要 · Abstract (English)

3D scene reconstruction and rendering are core tasks in computer vision, with applications spanning industrial monitoring, robotics, and autonomous driving. Recent advances in 3D Gaussian Splatting (GS) and its variants have achieved impressive rendering fidelity while maintaining high computational and memory efficiency. However, conventional vision-based GS pipelines typically rely on a sufficient number of camera views to initialize the Gaussian primitives and train their parameters, typically incurring additional processing cost during initialization while falling short in conditions where visual cues are unreliable, such as adverse weather, low illumination, or partial occlusions. To cope with these challenges, and motivated by the robustness of radio-frequency (RF) signals to weather, lighting, and occlusions, we introduce a multimodal framework that integrates RF sensing, such as automotive radar, with GS-based rendering as a more efficient and robust alternative to vision-only GS rendering. The proposed approach enables efficient depth prediction from only sparse RF-based depth measurements, yielding a high-quality 3D point cloud for initializing Gaussian functions across diverse GS architectures. Numerical tests demonstrate the merits of judiciously incorporating RF sensing into GS pipelines, achieving high-fidelity 3D scene rendering driven by RF-informed structural accuracy.

3D重建多模态雷达感知高斯泼溅

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。