用多摄像头无标定建模3D空间占用,免人工标注,跨数据集通用。
Visual Implicit Geometry Transformer for Autonomous Driving
- 基于多视角图像自监督学习,构建无需标定的连续3D占用场模型。
- 在5个数据集混合训练下,点云重建性能超越所有基线,平均排名最优。
- 适合追求少标注、跨传感器适配的自动驾驶几何建模场景。
我们提出视觉隐式几何变换器(ViGT),一种用于自动驾驶的几何建模方法,可从环视相机阵列中估计连续的3D占用场。ViGT迈向自动驾驶的基础几何模型,强调可扩展性、架构简洁性和对多种传感器配置的泛化能力。其核心是无标定架构,使单一模型可适应不同传感器布局。与聚焦像素对齐预测的通用几何基础模型不同,ViGT在鸟瞰图(BEV)中估计连续3D占用场,满足领域特定需求。该模型自然地将多视角图像融合至统一度量坐标系,为多个几何任务提供统一表示。不同于多数现有占用模型,我们采用自监督训练,利用同步的图像-LiDAR数据对,避免昂贵的人工标注。我们在五个大规模自动驾驶数据集(NuScenes、Waymo、NuPlan、ONCE、Argoverse)的混合数据上训练模型,并在点云重建任务上达到当前最优表现,所有对比基线中平均排名最高。进一步在Occ3D-nuScenes基准测试中,ViGT性能与有监督方法相当。源代码已公开于 https://github.com/whesense/ViGT。
原文摘要 · Abstract (English)
We introduce the Visual Implicit Geometry Transformer (ViGT), an autonomous driving geometric model that estimates continuous 3D occupancy fields from surround-view camera rigs. ViGT represents a step towards foundational geometric models for autonomous driving, prioritizing scalability, architectural simplicity, and generalization across diverse sensor configurations. Our approach achieves this through a calibration-free architecture, enabling a single model to adapt to different sensor setups. Unlike general-purpose geometric foundational models that focus on pixel-aligned predictions, ViGT estimates a continuous 3D occupancy field in a birds-eye-view (BEV) addressing domain-specific requirements. ViGT naturally infers geometry from multiple camera views into a single metric coordinate frame, providing a common representation for multiple geometric tasks. Unlike most existing occupancy models, we adopt a self-supervised training procedure that leverages synchronized image-LiDAR pairs, eliminating the need for costly manual annotations. We validate the scalability and generalizability of our approach by training our model on a mixture of five large-scale autonomous driving datasets (NuScenes, Waymo, NuPlan, ONCE, and Argoverse) and achieving state-of-the-art performance on the pointmap estimation task, with the best average rank across all evaluated baselines. We further evaluate ViGT on the Occ3D-nuScenes benchmark, where ViGT achieves comparable performance with supervised methods. The source code is publicly available at \href{https://github.com/whesense/ViGT}{https://github.com/whesense/ViGT}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。