通过结构化分块降低3D模型计算开销,实现高效可扩展的场景重建。
S-VGGT: Structure-Aware Subscene Decomposition for Scalable 3D Foundation Models
- 基于场景图将密集帧划分为少量子场景,从结构层面消除冗余。
- 子场景共享参考帧,使全局注意力成本降至原水平的1/40以下。
- 可与令牌级加速方法无缝结合,适合大规模3D建模任务。
前馈3D基础模型面临的核心挑战是全局注意力带来的二次计算开销,导致输入长度增加时严重限制可扩展性。现有加速方法如令牌合并虽在令牌层提供局部节省,但需进行最近邻搜索,引入额外开销。这些方法未能解决密集采集数据中占主导地位的结构性冗余问题。本文提出S-VGGT,一种在结构帧层面消除冗余的新方法。首先利用初始特征构建稠密场景图,刻画结构冗余并指导后续场景分割;基于该图,将帧软分配至少量子场景,保证组间平衡及平滑几何过渡。核心创新在于设计子场景共享同一参考帧,建立并行几何桥梁,实现独立高效处理且无需显式对齐。此结构重组从根本上削减全局注意力成本。关键优势在于S-VGGT与令牌级加速方法完全正交,可无缝组合,实现复合加速而不损失重建保真度。代码已开源:https://github.com/Powertony102/S-VGGT。
原文摘要 · Abstract (English)
Feed-forward 3D foundation models face a key challenge: the quadratic computational cost introduced by global attention, which severely limits scalability as input length increases. Concurrent acceleration methods, such as token merging, operate at the token level. While they offer local savings, the required nearest-neighbor searches introduce undesirable overhead. Consequently, these techniques fail to tackle the fundamental issue of structural redundancy dominant in dense capture data. In this work, we introduce \textbf{S-VGGT}, a novel approach that addresses redundancy at the structural frame level, drastically shifting the optimization focus. We first leverage the initial features to build a dense scene graph, which characterizes structural scene redundancy and guides the subsequent scene partitioning. Using this graph, we softly assign frames to a small number of subscenes, guaranteeing balanced groups and smooth geometric transitions. The core innovation lies in designing the subscenes to share a common reference frame, establishing a parallel geometric bridge that enables independent and highly efficient processing without explicit geometric alignment. This structural reorganization provides strong intrinsic acceleration by cutting the global attention cost at its source. Crucially, S-VGGT is entirely orthogonal to token-level acceleration methods, allowing the two to be seamlessly combined for compounded speedups without compromising reconstruction fidelity. Code is available at https://github.com/Powertony102/S-VGGT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。