arXiv:2603.20284cs.CVcs.GR2026-03中稿 · CVPR被引 5

用时空感知压缩技术,让3D实时重建内存降10倍、速度提4倍。

STAC: Plug-and-Play Spatio-Temporal Aware Cache Compression for Streaming 3D Reconstruction

  • 通过注意力衰减保留关键时序帧,避免早期缓存淘汰
  • 将空间冗余信息压缩为体素对齐表示,大幅节省内存
  • 分块多帧联合优化,提升画面连贯性与显卡效率

从流式输入中进行在线3D重建需要兼顾长期时间一致性与高效内存使用。尽管因果型VGGT通过键值(KV)缓存机制应对此挑战,但缓存随数据流长度线性增长,造成严重内存瓶颈。在有限内存预算下,早期缓存剔除显著降低重建质量与时间一致性。本文观察到:在3D重建任务中,因果变换器的注意力具有内在的时空稀疏性。基于此,提出STAC——一种面向大规模因果变换器的时空感知缓存压缩框架。其包含三个核心组件:(1) 工作时序标记缓存机制,利用衰减累积注意力得分保留长期有效标记;(2) 长期空间标记缓存方案,将空间冗余标记压缩为体素对齐表示以实现内存高效存储;(3) 块级多帧优化策略,联合处理连续帧以增强时间一致性与GPU效率。大量实验表明,STAC在保持最优重建质量的同时,内存消耗降低近10倍,推理加速4倍,显著提升流式场景下实时3D重建的可扩展性。

原文摘要 · Abstract (English)

Online 3D reconstruction from streaming inputs requires both long-term temporal consistency and efficient memory usage. Although causal variants of VGGT address this challenge through a key-value (KV) cache mechanism, the cache grows linearly with the stream length, creating a major memory bottleneck. Under limited memory budgets, early cache eviction significantly degrades reconstruction quality and temporal consistency. In this work, we observe that attention in causal transformers for 3D reconstruction exhibits intrinsic spatio-temporal sparsity. Based on this insight, we propose STAC, a Spatio-Temporally Aware Cache Compression framework for streaming 3D reconstruction with large causal transformers. STAC consists of three key components: (1) a Working Temporal Token Caching mechanism that preserves long-term informative tokens using decayed cumulative attention scores; (2) a Long-term Spatial Token Caching scheme that compresses spatially redundant tokens into voxel-aligned representations for memory-efficient storage; and (3) a Chunk-based Multi-frame Optimization strategy that jointly processes consecutive frames to improve temporal coherence and GPU efficiency. Extensive experiments show that STAC achieves state-of-the-art reconstruction quality while reducing memory consumption by nearly 10x and accelerating inference by 4x, substantially improving the scalability of real-time 3D reconstruction in streaming settings.

3D重建缓存压缩时空建模流式处理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。