用缓存关键值对实现实时3D姿态追踪，速度提升15倍
KV-Tracker: Real-Time Pose Tracking with Transformers
- 通过缓存全局自注意力的键值对，仅用少量关键帧构建场景表示
- 在多个数据集上实现约27帧/秒的实时追踪，精度不降且无漂移
- 适用于无深度信息的物体与场景追踪，可适配任意现成多视角网络
多视角3D几何网络虽具强大先验，但难以满足实时应用需求。本文提出一种新方法，使这类网络可用于在线追踪，实现单目RGB视频下的6-自由度姿态追踪与在线场景/物体重建。通过快速选择并管理图像作为关键帧，利用$π^3$结构实现全双向注意力，再将全局自注意力块的键值（KV）对缓存，作为唯一场景表征用于在线追踪。该策略使推理速度最高提升15倍，且避免漂移与灾难性遗忘。该缓存机制模型无关，无需重训练即可应用于其他现成多视角网络。在TUM RGB-D、7-Scenes、Arctic和OnePose数据集上的实验表明，系统性能优异，同时保持高达~27帧/秒的高帧率。
原文摘要 · Abstract (English)
Multi-view 3D geometry networks offer a powerful prior but are prohibitively slow for real-time applications. We propose a novel way to adapt them for online use, enabling real-time 6-DoF pose tracking and online reconstruction of objects and scenes from monocular RGB videos. Our method rapidly selects and manages a set of images as keyframes to map a scene or object via $π^3$ with full bidirectional attention. We then cache the global self-attention block's key-value (KV) pairs and use them as the sole scene representation for online tracking. This allows for up to $15\times$ speedup during inference without the fear of drift or catastrophic forgetting. Our caching strategy is model-agnostic and can be applied to other off-the-shelf multi-view networks without retraining. We demonstrate KV-Tracker on both scene-level tracking and the more challenging task of on-the-fly object tracking and reconstruction without depth measurements or object priors. Experiments on the TUM RGB-D, 7-Scenes, Arctic and OnePose datasets show the strong performance of our system while maintaining high frame-rates up to ${\sim}27$ FPS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。