提出新缓存机制，让视觉语言导航模型更高效实时运行。
VLN-Cache: Enabling Token Caching for VLN Models with Visual/Semantic Dynamics Awareness
- 根据视角和语义变化动态调整缓存内容，避免错误复用。
- 在模拟任务中实现1.52倍加速，导航成功率仍保持领先。
- 适合需要低延迟的智能机器人导航场景使用。
视觉-语言导航（VLN）越来越多依赖大型视觉语言模型，但其推理开销与实时部署需求冲突。令牌缓存是一种无需训练的高效策略，通过重用帧间稳定的视觉令牌来避免冗余计算。然而，现有方法假设相机静止且语义焦点固定，这与VLN的本质矛盾。我们识别出两种失效模式：(1) 视觉动态，视角变化导致令牌位置偏移，使逐位置匹配错误配对内容；(2) 语义动态，随着导航进展，令牌相关性发生变化，使缓存状态过时。我们提出VLN-Cache，一种兼顾视觉动态与语义动态感知的缓存框架，引入视图对齐重映射以恢复几何对应关系，并采用任务相关性显著性过滤器，在语义转换时拒绝复用。进一步设计分层自适应熵策略，平衡各层的复用预算。在R2R-CE仿真基准上的实验表明，该方法可实现最高1.52倍加速，同时保持竞争力的导航成功率。
原文摘要 · Abstract (English)
Vision-and-Language Navigation (VLN) increasingly relies on large vision-language models, but their inference cost conflicts with real-time deployment. Token caching is a promising training-free strategy that avoids redundant computation by reusing stable visual tokens across frames. However, existing methods assume a static camera and fixed semantic focus, assumptions that VLN fundamentally violates. We identify two failure modes: (1) visual dynamics, where viewpoint shift displaces token positions across frames, causing position-wise matching to pair misaligned content; (2) semantic dynamics, where token relevance shifts across task stages as navigation progresses, making cached states stale. We propose VLN-Cache, a visual-dynamic-aware and semantic-dynamic-aware caching framework that introduces view-aligned remapping to recover geometric correspondences and a task-relevance saliency filter to veto reuse at semantic transitions. A layer-adaptive entropy policy further balances the per-layer reuse budget. Experiments on the R2R-CE simulation benchmark show up to 1.52x speedup while maintaining competitive navigation success rates.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。