用视觉序列直接重建带尺度的3D场景，不依赖相机参数。
DVGT: Driving Visual Geometry Transformer
- 用Transformer融合多视角图像，自适应推断几何关系。
- 在nuScenes等5个数据集上优于现有方法，精度提升显著。
- 适合无标定相机的自动驾驶场景，无需外部传感器校准。
从视觉输入中感知与重建3D场景几何对自动驾驶至关重要。然而，尚缺乏能适配不同场景和相机配置的驾驶导向密集几何感知模型。为此，我们提出驾驶视觉几何变换器（DVGT），通过一系列未标定的多视角视觉输入重建全局稠密3D点云地图。首先使用DINO主干提取每张图像的视觉特征，再通过交替的视图内局部注意力、跨视图空间注意力与跨帧时间注意力，推断图像间的几何关系。随后，采用多头解码器在首帧自身坐标系下生成全局点云地图，并预测各帧的自身位姿。与依赖精确相机参数的传统方法不同，DVGT无需显式3D几何先验，可灵活处理任意相机配置。该模型直接从图像序列预测度量尺度的几何结构，无需与外部传感器进行后处理对齐。在nuScenes、OpenScene、Waymo、KITTI和DDAD等多个驾驶数据集混合训练下，DVGT在多种场景中显著超越现有模型。代码已开源：https://github.com/wzzheng/DVGT。
原文摘要 · Abstract (English)
Perceiving and reconstructing 3D scene geometry from visual inputs is crucial for autonomous driving. However, there still lacks a driving-targeted dense geometry perception model that can adapt to different scenarios and camera configurations. To bridge this gap, we propose a Driving Visual Geometry Transformer (DVGT), which reconstructs a global dense 3D point map from a sequence of unposed multi-view visual inputs. We first extract visual features for each image using a DINO backbone, and employ alternating intra-view local attention, cross-view spatial attention, and cross-frame temporal attention to infer geometric relations across images. We then use multiple heads to decode a global point map in the ego coordinate of the first frame and the ego poses for each frame. Unlike conventional methods that rely on precise camera parameters, DVGT is free of explicit 3D geometric priors, enabling flexible processing of arbitrary camera configurations. DVGT directly predicts metric-scaled geometry from image sequences, eliminating the need for post-alignment with external sensors. Trained on a large mixture of driving datasets including nuScenes, OpenScene, Waymo, KITTI, and DDAD, DVGT significantly outperforms existing models on various scenarios. Code is available at https://github.com/wzzheng/DVGT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。