一网打尽3D场景所有关键属性，高效且效果领先。
VGGT: Visual Geometry Grounded Transformer
- 用前馈网络统一预测相机参数、点云、深度图和3D轨迹
- 单张图重建不到1秒，多任务性能超越需后处理的模型
- 适合需要快速3D重建与下游任务增强的研究者
我们提出VGGT，一种前馈神经网络，可直接从一张、几张或上百张图像视图中推断场景的所有关键3D属性，包括相机参数、点图、深度图和3D点轨迹。该方法突破了传统3D计算机视觉中模型局限于单一任务的瓶颈，兼具简洁与高效：单图重建耗时不足1秒，且在多项3D任务上达到当前最优表现，涵盖相机参数估计、多视角深度估计、密集点云重建和3D点跟踪。此外，将预训练的VGGT作为特征主干，显著提升非刚性点跟踪和前馈式新视角合成等下游任务性能。代码与模型已公开于https://github.com/facebookresearch/vggt。
原文摘要 · Abstract (English)
We present VGGT, a feed-forward neural network that directly infers all key 3D attributes of a scene, including camera parameters, point maps, depth maps, and 3D point tracks, from one, a few, or hundreds of its views. This approach is a step forward in 3D computer vision, where models have typically been constrained to and specialized for single tasks. It is also simple and efficient, reconstructing images in under one second, and still outperforming alternatives that require post-processing with visual geometry optimization techniques. The network achieves state-of-the-art results in multiple 3D tasks, including camera parameter estimation, multi-view depth estimation, dense point cloud reconstruction, and 3D point tracking. We also show that using pretrained VGGT as a feature backbone significantly enhances downstream tasks, such as non-rigid point tracking and feed-forward novel view synthesis. Code and models are publicly available at https://github.com/facebookresearch/vggt.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。