用3D几何信息动态清理冗余帧，让单目视频重建更快更省内存。
GHOST: Geometry-Hierarchical Online Streaming Token Eviction for Efficient 3D Reconstruction
- 基于3D几何输出设计分层重要性评分，智能识别可删帧
- 缓存大小减半，推理速度提升1.75倍，重建质量几乎不变
- 无需训练，适合实时3D重建系统部署
从长序列单目视频进行流式3D重建需要维护一个随序列长度线性增长的键值（KV）缓存，造成严重内存瓶颈。现有方法要么固定截断缓存导致重建质量下降，要么依赖与3D场景结构无关的注意力分数启发式策略，无法保留几何上有价值的标记。为此，我们提出GHOST（Geometry-Hierarchical Online Streaming Token Eviction），一种无需训练的KV缓存管理框架，利用模型自身的3D几何输出在线剔除冗余标记。GHOST引入三项相互增强的创新：分层双级重要性评分机制、保护特殊标记免于剔除的特权机制，以及基于余弦相似度的逐层预算分配。在多个基准测试中，GHOST在将KV缓存减少近一半的同时保持优异重建质量，并实现比当前最优方法快1.75倍的推理速度。代码已开源：https://github.com/lokiniuniu/GHOST。
原文摘要 · Abstract (English)
Streaming 3D reconstruction from long monocular video sequences requires maintaining a key-value (KV) cache that grows linearly with sequence length, creating a severe memory bottleneck. Existing approaches either truncate the cache to a fixed set of anchor frames, leading to reconstruction quality degradation, or rely on attention-score heuristics that are agnostic to 3D scene structure, failing to preserve geometrically valuable tokens. To address these problems, we present GHOST (Geometry-Hierarchical Online Streaming Token Eviction), a training-free KV cache management framework that exploits the model's own 3D geometry outputs to evict redundant tokens online. GHOST introduces three mutually reinforcing innovations: a hierarchical dual-level importance scoring scheme, a privilege mechanism that protects special tokens from eviction, and a cosine-similarity-guided layer-wise budget allocation. Experiments on various benchmarks show that GHOST preserves excellent reconstruction quality while cutting the KV cache by nearly half and delivering 1.75x faster inference compared to state-of-the-art methods. Our code is available at https://github.com/lokiniuniu/GHOST.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。