arXiv:2511.22264cs.CV2025-11被引 2

针对多摄像头自动驾驶的几何重建难题,提出新模型提升精度与效率。

DriveVGGT: Calibration-Constrained Visual Geometry Transformers for Multi-Camera Autonomous Driving

  • 引入时序视频注意力模块,解决多视角重叠少的问题。
  • 利用相机标定信息实现绝对尺度重建,精度显著提升。
  • 适合自动驾驶场景,尤其长序列下表现更优。

前馈重建技术快速发展,视觉几何接地变换器(VGGT)成为重要基线。然而直接应用于自动驾驶(AD)时,难以捕捉三个领域特有先验:(i) 空间重叠稀疏:为满足360°覆盖且受预算限制,多视角摄像头重叠极小,导致全局注意力效率低下;(ii) 标定几何约束:自动驾驶数据可通过标定获取相机间绝对距离,但标准VGGT无法直接利用此信息进行绝对尺度重建;(iii) 外参刚性恒定:多视角相机相对位姿近似静态,即所有摄像头共享相同自运动。为弥合这些差距,本文提出DriveVGGT,一种显式融合上述先验的尺度感知重建框架,包含三个针对性组件:首先,针对稀疏空间重叠,引入时序视频注意力(TVA)模块,独立处理多摄像头视频;其次,针对标定几何约束,设计多相机一致性注意力(MCA)模块,结合标定信息与尺度头实现绝对尺度重建;最后,为利用外参刚性恒定,将VGGT解码过程重构为因子化顺序位姿头与自运动头。在多个自动驾驶数据集上,实验表明DriveVGGT在长序列场景中推理时间减少49.3%,同时优于原始VGGT的深度与位姿估计性能,并持续超越近期最先进方法。大量消融实验验证了各模块的有效性。

原文摘要 · Abstract (English)

Feed-forward reconstruction has been progressed rapidly, with the Visual Geometry Grounded Transformer (VGGT) being a notable baseline. However, directly applying VGGT to autonomous driving (AD) fails to capture three domain-specific priors: (i) Sparse Spatial Overlap: the overlap among mutli-view cameras is minimal due to $360^{\circ}$ coverage requirements under budget control, which renders global attention among all images inefficient; (ii) Calibrated Geometric Constraints: the absolute distance among cameras is generally accessible for AD data with calibration process before driving. Standard VGGT is unable to directly utilize such information for absolute scale scene reconstruction; (iii) Rigid Extrinsic Constancy: relative poses of multi-view cameras are approximately static, i.e., the ego-motion is the same for all cameras. To bridge these gaps, we propose DriveVGGT, a scale-aware reconstruction framework that explicitly integrates these priors through three targeted components. First, for the Sparse Spatial Overlap in (i), we introduce a Temporal Video Attention (TVA) module to process multi-camera videos independently. Second, for Calibrated Geometric Constraints in (ii), a Multi-camera Consistency Attention (MCA) module is designed to directly utilize the calibration information among cameras with a scale head for absolute scale scene reconstruction. Finally, to utilize Rigid Extrinsic Constancy in (iii), we reformulate the decoding process of VGGT into factorized sequential pose head and ego motion head. On AD datasets, experiments demonstrate that DriveVGGT reduces inference time by 49.3\% while improving depth and pose estimation compared to vanilla VGGT in long-sequence scenarios. It consistently outperforms recent SOTA variants. Meanwhile, extensive ablation studies verify the effectiveness of each devised module.

自动驾驶视觉几何多相机重建

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。