用历史导航经验增强大模型,让机器人零样本找物更准更快
TrajRAG: Retrieving Geometric-Semantic Experience for Zero-Shot Object Navigation

- 构建拓扑极坐标轨迹表示,压缩冗余观测数据
- 通过分层检索相似历史路径,提升导航决策准确率
- 适合需要长期积累经验的智能机器人导航任务
现有零样本物体导航方法依赖互联网文本中的常识知识,但这些知识缺乏具身3D体验。本文提出轨迹RAG(TrajRAG),一种检索增强生成框架,通过检索几何-语义经验来增强大模型推理。TrajRAG持续累积过往导航中的情景化观测,采用拓扑极坐标轨迹表示结构化这些数据,有效去除冗余;并设计分层分块机制,将相似轨迹归纳为统一摘要,支持粗到细的检索。导航时,候选前沿生成多个轨迹假设,向TrajRAG查询相似历史轨迹,辅助大模型选择目标点。新经验不断被整合进系统,实现终身导航经验积累。在MP3D、HM3D-v1和HM3D-v2上的实验表明,TrajRAG能有效检索相关几何-语义经验,显著提升零样本物体导航性能。
原文摘要 · Abstract (English)
Existing zero-shot Object Goal Navigation (ObjectNav) methods often exploit commonsense knowledge from large language or vision-language models to guide navigation. However, such knowledge arises from internet-scale text rather than embodied 3D experience, and episodic observations collected during navigation are typically discarded, preventing the accumulation of lifelong experience. To this end, we propose Trajectory RAG (TrajRAG), a retrieval-augmented generation framework that enhances large-model reasoning by retrieving geometric-semantic experiences. TrajRAG incrementally accumulates episodic observations from past navigation episodes. To structure these observations, we propose a topological-polar (topo-polar) trajectory representation that compactly encodes spatial layouts and semantic contexts, effectively removing redundancies in raw episodic observations. A hierarchical chunking structure further organizes similar topo-polar trajectories into unified summaries, enabling coarse-to-fine retrieval. During navigation, candidate frontiers generate multiple trajectory hypotheses that query TrajRAG for similar past trajectories, guiding large-model reasoning for waypoint selection. New experiences are continually consolidated into TrajRAG, enabling the accumulation of lifelong navigation experience. Experiments on MP3D, HM3D-v1, and HM3D-v2 show that TrajRAG effectively retrieves relevant geometric-semantic experiences and improves zero-shot ObjectNav performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。