arXiv:2506.05563cs.CV2025-06CVPR被引 13

用3D高斯点云动态建模,提升场景语义与运动预测精度。

VoxelSplat: Dynamic Gaussian Splatting as an Effective Loss for Occupancy and Flow Prediction

论文配图:VoxelSplat: Dynamic Gaussian Splatting as an Effective Loss for Occupancy and Flow Prediction
图 1 · 摘自论文原文
  • 通过2D投影提供额外语义监督,增强3D学习效果。
  • 自监督学习物体运动流,显著提升动态场景预测能力。
  • 可无缝集成现有模型,不增加推理耗时,适合实时应用。

基于摄像头的占用预测近年来聚焦于3D语义与场景流的联合预测,但因遮挡和动态环境不均衡等挑战而困难重重。本文分析了这些难题及其成因,提出名为VoxelSplat的新正则化框架。该框架利用3D高斯点云技术,在训练中将稀疏3D语义高斯点解码并投影至2D相机视图,从而在可见空间中引入额外监督信号,使2D标签有效提升3D语义学习;同时,利用预测的场景流建模高斯点运动,基于相邻帧标签实现自监督的场景流学习。该方法可无缝嵌入多种现有占用模型,提升性能且不增加推理时间。在多个基准数据集上的大量实验验证了其在语义占用与场景流估计上的有效性。

原文摘要 · Abstract (English)

Recent advancements in camera-based occupancy prediction have focused on the simultaneous prediction of 3D semantics and scene flow, a task that presents significant challenges due to specific difficulties, e.g., occlusions and unbalanced dynamic environments. In this paper, we analyze these challenges and their underlying causes. To address them, we propose a novel regularization framework called VoxelSplat. This framework leverages recent developments in 3D Gaussian Splatting to enhance model performance in two key ways: (i) Enhanced Semantics Supervision through 2D Projection: During training, our method decodes sparse semantic 3D Gaussians from 3D representations and projects them onto the 2D camera view. This provides additional supervision signals in the camera-visible space, allowing 2D labels to improve the learning of 3D semantics. (ii) Scene Flow Learning: Our framework uses the predicted scene flow to model the motion of Gaussians, and is thus able to learn the scene flow of moving objects in a self-supervised manner using the labels of adjacent frames. Our method can be seamlessly integrated into various existing occupancy models, enhancing performance without increasing inference time. Extensive experiments on benchmark datasets demonstrate the effectiveness of VoxelSplat in improving the accuracy of both semantic occupancy and scene flow estimation. The project page and codes are available at https://zzy816.github.io/VoxelSplat-Demo/.

3D重建高斯点云场景流自监督

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。