用时序尺度预测构建高效可控的3D占位世界模型
OccTENS: 3D Occupancy World Model via Temporal Next-Scale Prediction
- 将时序建模分解为尺度渐进生成与场景逐帧预测
- 在nuScenes数据集上实现更高占位质量与更快推理速度
- 支持车辆姿态控制,适合自动驾驶场景模拟
本文提出OccTENS,一种生成式3D占位世界模型,可在保持计算效率的同时实现高保真、可控制的长期占位生成。与视觉生成不同,占位世界模型需捕捉3D场景的细粒度几何结构和动态演化,对生成模型构成巨大挑战。现有基于自回归的方法虽能同时预测车辆运动与未来占位场景,但普遍存在效率低、长期生成中时间退化严重、缺乏可控性等问题。为此,我们重新将占位世界模型建模为时序下一尺度预测(TENS)任务,将时序序列建模分解为逐尺度空间生成与逐场景时间预测。通过TensFormer,OccTENS可灵活且可扩展地处理占位序列的时间因果关系与空间关联。为进一步增强姿态可控性,提出统一序列建模的全局姿态聚合策略,联合建模占位与自身运动。实验表明,OccTENS在nuScenes数据集上优于当前最先进方法,在占位质量与推理速度方面均表现更优。
原文摘要 · Abstract (English)
In this paper, we propose OccTENS, a generative occupancy world model that enables controllable, high-fidelity long-term occupancy generation while maintaining computational efficiency. Different from visual generation, the occupancy world model must capture the fine-grained 3D geometry and dynamic evolution of the 3D scenes, posing great challenges for the generative models. Recent approaches based on autoregression (AR) have demonstrated the potential to predict vehicle movement and future occupancy scenes simultaneously from historical observations, but they typically suffer from \textbf{inefficiency}, \textbf{temporal degradation} in long-term generation and \textbf{lack of controllability}. To holistically address these issues, we reformulate the occupancy world model as a temporal next-scale prediction (TENS) task, which decomposes the temporal sequence modeling problem into the modeling of spatial scale-by-scale generation and temporal scene-by-scene prediction. With a \textbf{TensFormer}, OccTENS can effectively manage the temporal causality and spatial relationships of occupancy sequences in a flexible and scalable way. To enhance the pose controllability, we further propose a holistic pose aggregation strategy, which features a unified sequence modeling for occupancy and ego-motion. Experiments show that OccTENS outperforms the state-of-the-art method with both higher occupancy quality and faster inference time.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。