LoGeR实现超长视频3D重建,突破注意力与记忆瓶颈。
LoGeR: Long-Context Geometric Reconstruction with Hybrid Memory
- 分块处理视频,结合双向先验与混合记忆增强块内精度。
- 训练128帧,推理可扩展至数千帧,KITTI上误差降低74%以上。
- 适合长时序3D重建任务,尤其适用于自动驾驶场景。
前馈几何基础模型在短窗口重建中表现优异,但扩展至分钟级视频时受限于二次注意力复杂度或循环设计的有效记忆容量。本文提出LoGeR(长上下文几何重建),一种新架构,可在无需后优化的情况下将密集3D重建扩展至极长序列。LoGeR以分块方式处理视频流,利用强双向先验实现高保真块内推理。为解决跨块连贯性难题,提出基于学习的混合记忆模块:包含参数化测试时训练(TTT)记忆以锚定全局坐标系并防止尺度漂移,以及非参数化滑动窗口注意力(SWA)机制以保留未压缩上下文,实现高精度邻接对齐。该记忆结构使LoGeR可在128帧序列上训练,并在推理时推广至数千帧。在标准基准和新重构的VBR数据集(最长达19,000帧)上评估,其显著优于现有前馈方法,在KITTI上降低ATE超过74%,实现前所未有的长程全局一致性重建。
原文摘要 · Abstract (English)
Feedforward geometric foundation models achieve strong short-window reconstruction, yet scaling them to minutes-long videos is bottlenecked by quadratic attention complexity or limited effective memory in recurrent designs. We present LoGeR (Long-context Geometric Reconstruction), a novel architecture that scales dense 3D reconstruction to extremely long sequences without post-optimization. LoGeR processes video streams in chunks, leveraging strong bidirectional priors for high-fidelity intra-chunk reasoning. To manage the critical challenge of coherence across chunk boundaries, we propose a learning-based hybrid memory module. This dual-component system combines a parametric Test-Time Training (TTT) memory to anchor the global coordinate frame and prevent scale drift, alongside a non-parametric Sliding Window Attention (SWA) mechanism to preserve uncompressed context for high-precision adjacent alignment. Remarkably, this memory architecture enables LoGeR to be trained on sequences of 128 frames, and generalize up to thousands of frames during inference. Evaluated across standard benchmarks and a newly repurposed VBR dataset with sequences of up to 19k frames, LoGeR substantially outperforms prior state-of-the-art feedforward methods--reducing ATE on KITTI by over 74%--and achieves robust, globally consistent reconstruction over unprecedented horizons.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。