用几何注意力机制实现高效稳定的实时3D重建
Geometric Context Transformer for Streaming 3D Reconstruction
- 设计几何上下文注意力，融合坐标锚点、位姿窗口和轨迹记忆
- 在518x378分辨率下稳定运行于20 FPS，支持超1万帧长序列
- 适合需要高精度与低延迟的实时3D场景重建应用
流式3D重建旨在从视频流中恢复相机位姿和点云等3D信息，要求几何精度高、时间一致性好且计算效率高。受同时定位与地图构建（SLAM）启发，我们提出LingBot-Map，一种基于几何上下文变压器（GCT）架构的前馈式3D基础模型，用于从流式数据重建场景。其核心在于精心设计的注意力机制：通过锚点上下文解决坐标定位问题，通过位姿参考窗口捕获密集几何线索，通过轨迹记忆实现长距离漂移校正。该设计在保持流式状态紧凑的同时，保留丰富几何上下文，使模型在518×378分辨率输入下，长时间序列（超过10,000帧）上实现约20 FPS的稳定高效推理。在多个基准测试中，该方法优于现有流式及迭代优化类方法。
原文摘要 · Abstract (English)
Streaming 3D reconstruction aims to recover 3D information, such as camera poses and point clouds, from a video stream, which necessitates geometric accuracy, temporal consistency, and computational efficiency. Motivated by the principles of Simultaneous Localization and Mapping (SLAM), we introduce LingBot-Map, a feed-forward 3D foundation model for reconstructing scenes from streaming data, built upon a geometric context transformer (GCT) architecture. A defining aspect of LingBot-Map lies in its carefully designed attention mechanism, which integrates an anchor context, a pose-reference window, and a trajectory memory to address coordinate grounding, dense geometric cues, and long-range drift correction, respectively. This design keeps the streaming state compact while retaining rich geometric context, enabling stable efficient inference at around 20 FPS on 518 x 378 resolution inputs over long sequences exceeding 10,000 frames. Extensive evaluations across a variety of benchmarks demonstrate that our approach achieves superior performance compared to both existing streaming and iterative optimization-based approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。