用高斯世界模型预测动态场景演化，提升自动驾驶3D占位预测精度。
GaussianWorld: Gaussian World Model for Streaming 3D Occupancy Prediction
- 将3D占位预测转为4D时序预测，分离静态场景、动态物体与新观测区域
- 在nuScenes上相比单帧模型提升2%以上mIoU，且不增加计算量
- 适合需要连续环境感知的自动驾驶系统，尤其关注场景演化建模
3D占位预测对自动驾驶至关重要，因其能全面感知周围环境。现有方法多通过融合前序帧特征来推断当前占位，但忽略了驾驶场景的连续性，未利用3D场景演变的强先验（如仅动态物体移动）。本文提出基于世界模型的框架，将3D占位预测重构为基于当前传感器输入的4D占位预测问题。将场景演化分解为三部分：1）静态场景的自车运动对齐；2）动态物体的局部运动；3）新观测区域的补全。采用高斯世界模型（GaussianWorld）显式建模这些先验，在3D高斯空间中结合当前RGB观测推断场景演化。在广泛使用的nuScenes数据集上验证，GaussianWorld在不引入额外计算开销的前提下，使单帧基线模型的mIoU提升超过2%。代码已开源。
原文摘要 · Abstract (English)
3D occupancy prediction is important for autonomous driving due to its comprehensive perception of the surroundings. To incorporate sequential inputs, most existing methods fuse representations from previous frames to infer the current 3D occupancy. However, they fail to consider the continuity of driving scenarios and ignore the strong prior provided by the evolution of 3D scenes (e.g., only dynamic objects move). In this paper, we propose a world-model-based framework to exploit the scene evolution for perception. We reformulate 3D occupancy prediction as a 4D occupancy forecasting problem conditioned on the current sensor input. We decompose the scene evolution into three factors: 1) ego motion alignment of static scenes; 2) local movements of dynamic objects; and 3) completion of newly-observed scenes. We then employ a Gaussian world model (GaussianWorld) to explicitly exploit these priors and infer the scene evolution in the 3D Gaussian space considering the current RGB observation. We evaluate the effectiveness of our framework on the widely used nuScenes dataset. Our GaussianWorld improves the performance of the single-frame counterpart by over 2% in mIoU without introducing additional computations. Code: https://github.com/zuosc19/GaussianWorld.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。