让3D重建速度突破瓶颈，千张图54秒完成
VGG-T$^3$: Offline Feed-Forward 3D Reconstruction at Scale
- 用测试时训练将可变长度表示压缩为固定大小MLP
- 1000张图仅需54秒，比基线快11.6倍
- 适合大规模离线3D重建与视觉定位任务
我们提出一种可扩展的3D重建模型，解决了离线前馈方法中计算与内存随输入图像数量呈二次增长的关键瓶颈。核心思路是将场景几何的可变长度键值（KV）表示通过测试时训练压缩为固定尺寸的多层感知机（MLP）。VGG-T³（基于视觉几何的测试时训练）在输入视图数上实现线性扩展，与在线模型相当，可在54秒内完成1000张图像的重建，相较依赖softmax注意力的基线提速11.6倍。由于保留全局场景聚合能力，其点云重建误差显著优于其他线性时间方法。此外，我们通过未见图像查询展示了模型的视觉定位能力。
原文摘要 · Abstract (English)
We present a scalable 3D reconstruction model that addresses a critical limitation in offline feed-forward methods: their computational and memory requirements grow quadratically w.r.t. the number of input images. Our approach is built on the key insight that this bottleneck stems from the varying-length Key-Value (KV) space representation of scene geometry, which we distill into a fixed-size Multi-Layer Perceptron (MLP) via test-time training. VGG-T$^3$ (Visual Geometry Grounded Test Time Training) scales linearly w.r.t. the number of input views, similar to online models, and reconstructs a $1k$ image collection in just $54$ seconds, achieving a $11.6\times$ speed-up over baselines that rely on softmax attention. Since our method retains global scene aggregation capability, our point map reconstruction error outperforming other linear-time methods by large margins. Finally, we demonstrate visual localization capabilities of our model by querying the scene representation with unseen images.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。