提出有限内存框架,让3D视觉流长期稳定感知
FrameVGGT: Coherence-Preserving Memory for Bounded Streaming Geometry
- 用帧级记忆单元和稀疏锚点,固定内存容量下保持几何连贯性
- 在长序列重建/深度估计/位姿估计中,精度与内存平衡更优
- 适合需要持续3D感知的自动驾驶、机器人等实时系统
流式视觉几何变换器如StreamVGGT能实现强大的在线3D感知,但其键值缓存(KV-cache)随长时间流数据无限增长,限制了实际部署。本文从内存组织角度研究有限内存下的流式几何推理:与语言建模中可按词元压缩信息不同,几何推断依赖多视角间连贯且互兼容的观测。在固定内存预算下,若仅孤立保存历史信息,会逐步破坏长时匹配与融合所需的几何上下文。为此,我们提出FrameVGGT——一种维持固定容量互补记忆单元的有限内存框架。每个记忆单元由紧凑的键空间原型总结的帧级键值段,以及用于持久长程参考的稀疏锚点层构成。在长序列3D重建、视频深度估计和相机位姿估计任务中,FrameVGGT在有限内存下实现了更优的精度-内存权衡,并显著提升了长时几何稳定性。
原文摘要 · Abstract (English)
Streaming Visual Geometry Transformers such as StreamVGGT enable strong online 3D perception, but their KV-cache grows unbounded over long streams, limiting practical deployment. We study bounded-memory streaming geometry from the perspective of memory organization: unlike language modeling, where useful information can often be compressed at token level, geometry-driven inference relies on coherent and mutually compatible observations across views. Under fixed memory budgets, retaining history as isolated entries can progressively fragment the geometric context needed for stable long-horizon matching and fusion. We therefore propose \textbf{FrameVGGT}, a bounded-memory framework that maintains a fixed-capacity set of complementary memory units for streaming geometry. In our implementation, each unit is instantiated as a frame-wise KV segment summarized by a compact key-space prototype, together with a sparse anchor tier for persistent long-range references. Across long-sequence 3D reconstruction, video depth estimation, and camera pose estimation, FrameVGGT achieves favorable accuracy--memory trade-offs under bounded budgets while maintaining more stable geometry over long streams.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。