arXiv:2508.09262cs.CVcs.LG2025-08ICCV被引 1

通过自适应推理提升视觉语言导航效率,计算量减半且性能不降。

Harnessing Input-Adaptive Inference for Efficient VLN

  • 按输入动态调整处理策略,分层优化空间、模型和时间效率。
  • 在7个基准上实现超过2倍的计算量减少,跨标准与连续环境有效。
  • 适合资源受限场景下的智能体部署,开源代码可复现。

视觉语言导航(VLN)领域正采用具备历史感知能力的多模态变换器模型,根据语言指令和导航历史预测最优动作。尽管性能显著提升,其庞大模型规模在计算资源有限的实际场景中成为瓶颈。本文提出一种新型输入自适应导航方法,以提升效率。首先发现现有自适应机制难以在不损失性能的前提下减少计算量。为此,我们在三个层面引入新算法:(1) 在空间层面,仅选择性处理智能体每步观察中的全景视图;(2) 在模型内部,基于重要性设计早期退出的自适应阈值;(3) 在时间层面,采用缓存机制避免重复处理已见过的视图。在七个VLN基准上的评估显示,对三种现成智能体,在标准与连续环境中均实现超过2×的计算量减少。代码公开于https://github.com/secure-ai-systems-group/adaptive-vision-and-language-navigation。

原文摘要 · Abstract (English)

An emerging paradigm in vision-and-language navigation (VLN) is the use of history-aware multi-modal transformer models. Given a language instruction, these models process observation and navigation history to predict the most appropriate action for an agent. While they have significantly improved performance, the scale of these models can be a bottleneck in practical settings with limited computational resources. In this work, we propose a novel input-adaptive navigation method to enhance VLN model efficiency. We first show that existing input-adaptive mechanisms fail to reduce computations without substantial performance degradation. To address this, we introduce three adaptive algorithms, each deployed at a different level: (1) To improve spatial efficiency, we selectively process panoramic views at each observation of an agent. (2) To improve intra-model efficiency, we propose importance-based adaptive thresholding for the early-exit methods. (3) To improve temporal efficiency, we implement a caching mechanism that prevents reprocessing of views previously seen by the agent. In evaluations on seven VLN benchmarks, we demonstrate over a 2$\times$ reduction in computation across three off-the-shelf agents in both standard and continuous environments. Our code is publicly available at https://github.com/secure-ai-systems-group/adaptive-vision-and-language-navigation.

视觉语言导航自适应推理高效模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。