提出记忆-执行-复盘框架，零样本导航成功率显著提升
MerNav: A Highly Generalizable Memory-Execute-Review Framework for Zero-Shot Object Goal Navigation
- 分层记忆+执行+复盘三模块协同决策
- 零样本下在4个数据集平均提效7%-8%
- 实机部署于人形机器人，兼顾性能与泛化
视觉语言导航（VLN）是具身智能的核心能力之一，也是亟待解决的关键挑战。现有方法在成功率（SR）和泛化能力之间难以兼顾：监督微调（SFT）方法通常成功率更高，而无训练（TF）方法泛化性更好。为此，我们提出一种记忆-执行-复盘（Memory-Execute-Review）框架，包含三个部分：分层记忆模块提供信息支持，执行模块负责常规决策与动作，复盘模块处理异常情况并纠正行为。我们在物体目标导航任务上验证了该框架的有效性。在4个数据集上，相较于所有基线方法，零样本（ZS）设置下平均成功率分别提升8%（HM3D_v0.1）和6%（HM3D_OVON）。在MP3D和HM3D_OVON数据集上，不仅超越所有TF方法，还超过所有SFT方法，在成功率上分别提升5%和2%，实现全面领先。此外，我们在人形机器人上部署了MerNav模型，并在真实世界中完成实验。
原文摘要 · Abstract (English)
Visual Language Navigation (VLN) is one of the fundamental capabilities for embodied intelligence and a critical challenge that urgently needs to be addressed. However, existing methods are still unsatisfactory in terms of both success rate (SR) and generalization: Supervised Fine-Tuning (SFT) approaches typically achieve higher SR, while Training-Free (TF) approaches often generalize better, but it is difficult to obtain both simultaneously. To this end, we propose a Memory-Execute-Review framework. It consists of three parts: a hierarchical memory module for providing information support, an execute module for routine decision-making and actions, and a review module for handling abnormal situations and correcting behavior. We validated the effectiveness of this framework on the Object Goal Navigation task. Across 4 datasets, our average SR achieved absolute improvements of 7% and 5% compared to all baseline methods under TF and Zero-Shot (ZS) settings, respectively. On the most commonly used HM3D_v0.1 and the more challenging open vocabulary dataset HM3D_OVON, the SR improved by 8% and 6%, under ZS settings. Furthermore, on the MP3D and HM3D_OVON datasets, our method not only outperformed all TF methods but also surpassed all SFT methods, achieving comprehensive leadership in both SR (5% and 2%) and generalization. Additionally, we deployed the MerNav model on the humanoid robot and conducted experiments in the real world. The project address is: https://qidekang.github.io/MerNav.github.io/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。