arXiv:2503.08485cs.CV2025-03被引 2

无需训练即可动态适配分辨率,实时生成3D占位图

Test-Time 3D Occupancy Prediction

  • 运行时用视觉基础模型构建可变分辨率3D高斯点云
  • 在nuScenes数据集上比现有方法快10倍且精度更高
  • 适合自动驾驶场景中快速部署新物体识别

自监督3D占位预测为理解复杂驾驶场景提供了有前景的解决方案,无需昂贵的3D标注。然而,训练密集的占位解码器以捕捉精细几何和语义特征可能需要数百小时的GPU时间,且模型训练后难以适应不同体素分辨率或新物体类别,需大量重训练。为此,我们提出一种实用且灵活的测试时占位预测框架TT-Occ。该方法通过集成视觉基础模型(VFMs),从原始传感器流中增量式构建、优化并体素化时序感知的3D高斯点云。3D高斯点云的灵活表示支持任意用户指定分辨率的体素化,而VFMs强大的泛化能力则实现无需任何网络训练或微调的准确感知与开放词汇识别。为验证框架的通用性与有效性,我们提出了两种变体:基于激光雷达的版本和以视觉为中心的版本,并在不同体素分辨率下对Occ3D-nuScenes和nuCraft基准进行了大量实验。结果表明,TT-Occ显著优于现有计算成本高昂的预训练自监督方法。代码已公开于https://github.com/Xian-Bei/TT-Occ。

原文摘要 · Abstract (English)

Self-supervised 3D occupancy prediction offers a promising solution for understanding complex driving scenes without requiring costly 3D annotations. However, training dense occupancy decoders to capture fine-grained geometry and semantics can demand hundreds of GPU hours, and once trained, such models struggle to adapt to varying voxel resolutions or novel object categories without extensive retraining. To overcome these limitations, we propose a practical and flexible test-time occupancy prediction framework termed TT-Occ. Our method incrementally constructs, optimizes, and voxelizes time-aware 3D Gaussians from raw sensor streams by integrating vision foundation models (VFMs) at runtime. The flexible representation of 3D Gaussians enables voxelization at arbitrary user-specified resolutions, while the strong generalization capability of VFMs supports accurate perception and open-vocabulary recognition without requiring any network training or fine-tuning. To validate the generality and effectiveness of our framework, we present two variants: a LiDAR-based version and a vision-centric version, and conduct extensive experiments on the Occ3D-nuScenes and nuCraft benchmarks under varying voxel resolutions. Experimental results show that TT-Occ significantly outperforms existing computationally expensive pretrained self-supervised counterparts. Code is available at https://github.com/Xian-Bei/TT-Occ.

3D占位自监督测试时优化视觉基础模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。