arXiv:2409.02561cs.AIcs.RO2024-09被引 6

让AI导航机器人像人一样持续学习新环境,不忘记旧知识。

Vision-Language Navigation with Continual Learning

  • 用双循环场景回放机制模拟大脑记忆巩固过程
  • 在新环境中实现快速适应,同时避免遗忘旧经验
  • 为视觉语言导航引入持续学习新范式,适合长期部署场景

视觉-语言导航(VLN)是嵌入式智能的关键领域,要求智能体根据自然语言指令在3D环境中导航。传统方法聚焦于提升环境理解与决策准确性,但在新环境中常表现不佳,主要因训练数据多样性不足。扩展数据集成本高昂且不现实。本文提出视觉-语言导航的持续学习范式(VLNCL),使智能体在增量学习新环境的同时保留已有知识。该范式通过构建环境记忆并提取相关知识,实现对新环境的快速适应。我们提出一种受大脑记忆回放机制启发的新型双循环场景回放方法(Dual-SR),结合多场景记忆缓冲区,高效组织与重播任务记忆,强化泛化能力并缓解灾难性遗忘。本工作首次将持续学习引入VLN智能体,建立了新的实验设置与评估指标。大量实验证明该方法显著优于现有持续学习与VLN方法,在持续学习能力上达到最先进水平,展现出快速适应与知识保留的潜力。

原文摘要 · Abstract (English)

Vision-language navigation (VLN) is a critical domain within embedded intelligence, requiring agents to navigate 3D environments based on natural language instructions. Traditional VLN research has focused on improving environmental understanding and decision accuracy. However, these approaches often exhibit a significant performance gap when agents are deployed in novel environments, mainly due to the limited diversity of training data. Expanding datasets to cover a broader range of environments is impractical and costly. We propose the Vision-Language Navigation with Continual Learning (VLNCL) paradigm to address this challenge. In this paradigm, agents incrementally learn new environments while retaining previously acquired knowledge. VLNCL enables agents to maintain an environmental memory and extract relevant knowledge, allowing rapid adaptation to new environments while preserving existing information. We introduce a novel dual-loop scenario replay method (Dual-SR) inspired by brain memory replay mechanisms integrated with VLN agents. This method facilitates consolidating past experiences and enhances generalization across new tasks. By utilizing a multi-scenario memory buffer, the agent efficiently organizes and replays task memories, thereby bolstering its ability to adapt quickly to new environments and mitigating catastrophic forgetting. Our work pioneers continual learning in VLN agents, introducing a novel experimental setup and evaluation metrics. We demonstrate the effectiveness of our approach through extensive evaluations and establish a benchmark for the VLNCL paradigm. Comparative experiments with existing continual learning and VLN methods show significant improvements, achieving state-of-the-art performance in continual learning ability and highlighting the potential of our approach in enabling rapid adaptation while preserving prior knowledge.

视觉语言导航持续学习记忆回放智能体

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。