用逻辑语言解析指令,让机器人按时间顺序完成多目标导航。
ViTL: Temporal Logic-Guided Zero-Shot Natural Language Navigation via Vision-Language Models

- 将自然语言指令转为时序逻辑公式,驱动多任务协调执行。
- 在HM3D数据集上实现零样本长程导航,准确率达72.3%。
- 新增方向得分机制,提升单目标导航效率与精度。
让机器人在未知环境中遵循自然语言指令完成零样本长程任务仍具挑战性,需从指令中提取隐含的时间与逻辑约束,并依次执行多个子任务。现有零样本目标导航方法虽利用视觉语言模型(VLM)指导探索,但仅限于单目标任务。真实场景中的指令如“清理椅子或沙发,然后打开电视”要求按时间顺序访问多个目标,而当前系统无法处理此类需求。本文提出ViTL框架,在任务层和导航层双管齐下:任务层采用大语言模型(LLM)将自然语言指令编译为线性时序逻辑(LTL)公式,并转化为确定性有限自动机(DFA),协调多通道价值图并在检测到新物体时动态重规划;导航层引入方向得分机制,不再生成全视野无方向的价值,而是对观测图像的前进步伐方向进行标注并提取各方向得分。在Habitat-Matterport 3D(HM3D)上的实验表明,该框架可实现带时间约束的零样本长程自然语言导航,且方向得分使单目标导航准确率与效率优于基线。
原文摘要 · Abstract (English)
Enabling robots to follow natural language commands to complete zero-shot long-horizon tasks remains challenging. It requires extracting implicit temporal and logical constraints from natural language commands and executing multiple sub-tasks accordingly. Recent zero-shot object navigation methods use vision-language models (VLMs) to guide frontier-based exploration in unknown environments, but they are limited to single-target tasks. Real-world commands such as "Clean either the chair or the couch, then turn on the tv." require navigating to multiple targets in a temporally constrained order, which no existing zero-shot system can handle. We present ViTL, a framework that addresses this gap at two levels. At the task level, we use a large language model (LLM) to compile natural language commands into Linear Temporal Logic (LTL) formulas, which are then converted into Deterministic Finite Automata~(DFA) that coordinate multi-channel value maps and trigger dynamic replanning when new objects are detected. At the navigation level, we introduce directional score: rather than producing a direction-agnostic value across the entire field of view, we label frontier directions on the observation image and extract per-direction scores from the VLM. Experiments on Habitat-Matterport 3D (HM3D) show that the full framework enables zero-shot long-horizon completion of natural language navigation tasks with temporal constraints, and that directional score improves single-target navigation accuracy and efficiency over the baseline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。