用稀疏局部场实现多传感器实时高保真模拟,兼顾速度与精度。
SaLF: Sparse Local Fields for Multi-Sensor Rendering in Real-Time
- 将体积表示为稀疏3D体素,每个体素含局部隐式场,支持光栅化与光线追踪
- 相机渲染超50帧/秒,激光雷达达600帧/秒,训练时间少于30分钟
- 兼容非针孔相机与旋转激光雷达,适合自动驾驶多传感器仿真
基于神经辐射场(NeRF)的光传感器模拟方法虽能高保真重建驾驶场景,但训练和渲染速度慢,难以扩展。3D高斯点云(3DGS)虽加速了训练与渲染,但仅适用于针孔相机,限制了多传感器仿真。此外,两者将表示与渲染绑定,缺乏互操作性。本文提出稀疏局部场(SaLF),一种统一的体积表示方法,支持光栅化与光线追踪,用于多传感器实时模拟。SaLF以稀疏3D体素为基本单元,每个体素包含局部隐式场,具备快速训练(<30分钟)与高效渲染能力(相机>50 FPS,LiDAR>600 FPS),支持自适应剪枝与加密,可处理大场景,并兼容非针孔相机与旋转激光雷达。实验表明,SaLF在真实感上媲美现有方法,同时显著提升效率与扩展性,推动自动驾驶仿真更规模化。
原文摘要 · Abstract (English)
High-fidelity sensor simulation of light-based sensors such as cameras and LiDARs is critical for safe and accurate autonomy testing. Neural radiance field (NeRF)-based methods that reconstruct sensor observations via ray-casting of implicit representations have demonstrated accurate simulation of driving scenes, but are slow to train and render, hampering scalability. 3D Gaussian Splatting (3DGS) has demonstrated faster training and rendering times through rasterization, but is primarily restricted to pinhole camera sensors, preventing usage for realistic multi-sensor autonomy evaluation. Moreover, both NeRF and 3DGS couple the representation with the rendering procedure (implicit networks for ray-based evaluation, particles for rasterization), preventing interoperability, which is key for general usage. In this work, we present Sparse Local Fields (SaLF), a novel volumetric representation that supports rasterization and raytracing for unified multi-sensor simulation. SaLF represents volumes as a sparse set of 3D voxel primitives, where each voxel is a local implicit field. SaLF has fast training ($<$30 min) and rendering capabilities (50+ FPS for camera and 600+ FPS for LiDAR), has adaptive pruning and densification to easily handle large scenes, and can support non-pinhole cameras and spinning LiDARs. We demonstrate that SaLF has similar realism as existing self-driving sensor simulation methods while improving efficiency and enhancing capabilities, enabling more scalable simulation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。