arXiv:2509.02560cs.CV2025-09被引 97

不训练即可加速3D视觉模型,解决长序列输入效率低问题

FastVGGT: Training-Free Acceleration of Visual Geometry Transformer

  • 通过无训练的令牌合并机制优化3D视觉架构
  • 1000张图像输入时速度提升4倍,且减少误差累积
  • 适合需要高效处理长序列3D数据的研究者

面向3D视觉的基础模型在3D感知任务中展现出强大能力,但面对长序列图像输入时仍存在推理效率低的问题。本文深入分析了当前先进的前馈式视觉几何模型VGGT,发现其主要瓶颈在于注意力图中的令牌坍塌现象。基于此,我们探索在前馈视觉几何模型中应用令牌合并技术。由于3D模型的架构特性和任务需求,直接套用现有合并方法困难。为此,我们提出FastVGGT,首次在3D领域通过无训练机制实现令牌合并,设计了适配3D架构与任务的特殊令牌划分策略,在消除冗余计算的同时保持VGGT强大的重建能力。在多个3D几何基准测试中,实验验证了该方法的有效性。特别地,在1000张输入图像场景下,FastVGGT相比VGGT实现4倍加速,同时缓解了长序列中的误差累积问题。这些结果表明,令牌合并是构建可扩展3D视觉系统的一种有效途径。代码已开源:https://mystorm16.github.io/fastvggt/

原文摘要 · Abstract (English)

Foundation models for 3D vision have recently demonstrated remarkable capabilities in 3D perception. However, scaling these models to long-sequence image inputs remains a significant challenge due to inference-time inefficiency. In this work, we present a detailed analysis of VGGT, a state-of-the-art feed-forward visual geometry model and identify its primary bottleneck. Visualization further reveals a token collapse phenomenon in the attention maps. Motivated by these findings, we explore the potential of token merging in the feed-forward visual geometry model. Owing to the unique architectural and task-specific properties of 3D models, directly applying existing merging techniques proves challenging. To this end, we propose FastVGGT, which, for the first time, leverages token merging in the 3D domain through a training-free mechanism for accelerating VGGT. we devise a unique token partitioning strategy tailored to 3D architectures and tasks, effectively eliminating redundant computation while preserving VGGT's powerful reconstruction capacity. Extensive experiments on multiple 3D geometry benchmarks validate the effectiveness of our approach. Notably, with 1000 input images, FastVGGT achieves a 4x speedup over VGGT while mitigating error accumulation in long-sequence scenarios. These findings underscore the potential of token merging as a principled solution for scalable 3D vision systems. Code is available at: https://mystorm16.github.io/fastvggt/.

3D视觉模型加速令牌合并

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。