让机器人边走边记,用记忆反思决策,提升从未见过建筑中的寻物成功率。
Hierarchical Fast-Slow ReAct Agent for Zero-Shot Object-Goal Navigation

- 构建快慢双层系统:快层实时导航,慢层基于记忆推理
- 在HM3D上达68.75%成功率,超越现有零样本方法
- 适合需自主推理的复杂环境导航任务
零样本物体目标导航要求机器人在从未进入过的建筑中找到指定类别的物体。当前方法通过视觉语言价值图对前沿进行评分,每次决策仅依赖当前视图,且不保留历史证据。典型系统在感知-动作循环中固定频率调用大模型,无法回溯过去观察,失败也无应对机制。本文将机器人已见内容作为反思对象,提出分层快速-慢速代理:每一步保持价值图控制器运行,移动时记录带坐标锚定的记忆——包含房间类型与确认物体实例的语义网格,以及有限的位姿标记关键帧。视觉语言模型在写入前筛选候选检测。推理层以有限的‘思考-检索-行动’循环读取记忆,仅在文本无法区分时调用第一人称视图。每次调用和每轮运行均有上限,无需调用的第一层级解决多数卡顿问题,任何失败均返回快速控制器。系统在HM3D v1验证集达68.75%成功率,在MP3D上达47.29%,为当前对比方法最高。相比仅用argmax选择远端,通过推理而非单纯选优损失3.40成功率点(95%置信区间[1.70, 5.05]),但对所有前沿都推理也无法挽回损失。
原文摘要 · Abstract (English)
Zero-shot object-goal navigation (ZSON) requires a robot to find a named object category in a building it has never entered. The prevailing approach scores frontiers with a vision-language value map: every decision is another argmax over the map as it currently stands, and the evidence behind that score is discarded the moment it is taken. Systems that place a large vision-language model inside the perception-action loop typically query it on a fixed schedule from the current view alone; a room the robot walked through minutes earlier is never reconsidered, and a failed call has no defined fallback. We turn what the robot has already seen into the object of deliberation. Our hierarchical fast-slow agent leaves the value-map controller running at every step and writes a coordinate-anchored memory as it moves: a semantic grid of room types and confirmed object instances, together with a bounded store of pose-tagged keyframes. A VLM screens each candidate detection before it is written. A deliberative layer reads this memory in a bounded reason-retrieve-act loop. It wakes on structural events the reactive layer computes, reasons first over text, and recalls a first-person view only for candidates that text alone cannot separate. Per-invocation and per-run caps bound its calls, a call-free first tier resolves the most frequent stall, and any failure returns control to the reactive controller. Our system reaches 68.75% SR on HM3D v1 val and 47.29% on MP3D val, the highest success rate among the zero-shot methods compared here. Choosing among far frontiers by argmax instead of deliberating costs 3.40 SR points in a paired comparison over all 2000 HM3D episodes (95% CI [1.70, 5.05]); deliberating over every frontier does not recover them.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。