arXiv:2607.06537cs.RO2026-07

用大模型实现零样本末段导航,让机器人精准抵达可操作位置。

UniLM-Nav: A Unified Framework for Zero-Shot Last-Mile Navigation

论文配图:UniLM-Nav: A Unified Framework for Zero-Shot Last-Mile Navigation
图 1 · 摘自论文原文
  • 用统一框架分解导航任务,全靠大模型理解视觉与任务意图。
  • 在OVMM基准上超越前人方法3.13个百分点,实测有效。
  • 适合做开放词汇的移动操作,尤其适用于真实机器人部署。

移动操作要求机器人导航至目标物体或容器并执行相应操作,但仅到达附近并不保证具备可操作的基座姿态,这一问题称为末段导航。以往方法依赖人工标注姿态或特定任务训练,难以扩展到开放词汇且具精细空间约束的场景。本文提出UniLM-Nav,一种统一的零样本开放词汇末段导航框架。该框架将末段导航分解为视图选择、任务条件下的可操作性定位以及几何感知的基座姿态推理,均由共享的多模态大语言模型(MLLM)完成。具体而言,UniLM-Nav首先从近期观测中选取最能体现目标物体或容器的参考视图;接着在该视图中定位任务相关的可操作点,并将其转换至机器人坐标系;最后,在已定位可操作点、任务上下文及机器人几何信息的条件下,推断出适合操作的基座姿态。我们在OVMM基准上评估了UniLM-Nav,其性能优于当前最优方法MoTo,提升3.13个百分点。分析表明各模块对最终性能至关重要,且MLLM选型影响显著。我们进一步在Unitree B2四足机器人搭载6-DoF Unitree Z1机械臂上部署,验证了其在真实移动操作任务中的可行性。

原文摘要 · Abstract (English)

Mobile manipulation requires a robot to navigate to a target object or receptacle and then perform intended manipulation. However, reaching the vicinity of the target does not guarantee a manipulation-ready base pose, a problem known as last-mile navigation. Prior methods for last-mile navigation either rely on manual pose annotation or task-specific training, limiting their scalability to open-vocabulary settings with fine-grained spatial constraints. We propose UniLM-Nav, a unified framework for zero-shot open-vocabulary last-mile navigation. UniLM-Nav decomposes last-mile navigation into view selection, task-conditioned affordance grounding, and geometry-aware base-pose reasoning, all resolved with a shared multimodal large language model (MLLM) backend. Specifically, UniLM-Nav first selects a reference view that best captures the target object or receptacle from recently collected observations. It then grounds task-relevant affordance point in the selected view and lifts the result into the robot-centric coordinate frame. Finally, conditioned on the grounded affordance, task context, and robot geometry, it infers a manipulation-ready base pose for the robot. We evaluate UniLM-Nav on the OVMM benchmark, where it outperforms the previous state-of-the-art method, MoTo, by 3.13 percentage points. Analyses show that the components of our method are crucial to final performance, and that the choice of MLLM also has a substantial effect. We further deploy UniLM-Nav on a Unitree B2 quadruped robot with a 6-DoF Unitree Z1 manipulator, validating its applicability to real-world mobile manipulation tasks.

末段导航大模型移动操作零样本

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。