arXiv:2607.13624cs.ROcs.AI2026-07

让机器人听懂人话,用视觉感知自动导航到目标位置

From Language to Navigation Goals: A Vision-Language Approach for Semantic Navigation of Mobile Robots Using RGB-D Perception

论文配图:From Language to Navigation Goals: A Vision-Language Approach for Semantic Navigation of Mobile Robots Using RGB-D Perception
图 1 · 摘自论文原文
  • 将自然语言指令转为导航动作,结合视觉与语义理解
  • 在仿真和真实环境均实现90%以上成功率的精准导航
  • 模块化设计适配多种机器人平台,适合人机交互研究者

自然语言交互为非专业用户提供了与机器人平台沟通的直观方式。然而,将用户请求转化为可执行的导航动作仍具挑战性,需融合语言理解、环境感知与自主导航。本文提出一种基于语言驱动的导航框架,使移动机器人能够解析自然语言指令,自主导航至目标位置。该框架由模块化ROS 2组件构成,协同将自然语言指令转换为导航动作。给定指向环境中目标的自然语言请求(如“去邮箱处”),系统通过RGB-D数据识别目标物体并估计其位置,生成导航目标,并由ROS 2 Nav2导航栈执行。基于ROS 2的实现支持跨平台移植,仅需配置对应话题与服务。在TurtleBot3 Waffle和Unitree Go2机器人上,搭载RealSense相机,在仿真与真实场景中验证了本系统对直接指令和上下文请求的准确解析能力,生成有意义的语言反馈,并成功抵达目标。结果表明,结合语义感知与自主导航,可实现直观的人机交互范式。代码将在论文录用后开源。

原文摘要 · Abstract (English)

Natural language interaction provides an intuitive way for non-expert users to communicate with robotic platforms. However, transforming user requests into executable navigation actions remains a challenging task, requiring the integration of language understanding, environment perception, and autonomous navigation. This work presents a language-driven navigation framework that enables mobile robots to interpret user requests in natural language to move the robot to a destination and autonomously navigate towards it. The framework is composed of modular ROS 2 components that cooperate to transform natural language instructions into navigation actions. Given a natural language request referring to a target in the environment (e.g., "go to the mail box"), the system identifies the referenced object, estimates its position using RGB-D data, and generates a navigation goal, which is then executed through the ROS 2 Nav2 navigation stack. The ROS 2-based implementation facilitates portability across different robotic platforms, requiring only the configuration of the corresponding topics and services. The system is evaluated in both simulation and real-world scenarios using a TurtleBot3 Waffle and a Unitree Go2 robot with a RealSense camera. Experimental results show that the framework successfully interprets both direct commands and contextual requests, generates meaningful natural-language feedback, and navigates towards the desired target. These results demonstrate the feasibility of combining semantic perception and autonomous navigation to provide an intuitive human-robot interaction paradigm. Code will be released as open source upon acceptance.

语义导航人机交互ROS2RGB-D

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。