arXiv:2409.10283cs.ROcs.AI2024-09中稿 · ICRA被引 2

让无人机听懂指令的同时实时避障,安全成功率提升超60%。

ASMA: An Adaptive Safety Margin Algorithm for Vision-Language Drone Navigation via Scene-Aware Control Barrier Functions

  • 基于场景感知的自适应安全边界算法,动态评估环境风险
  • 在仿真中使导航成功率提升64%-67%,路径仅增加1.4%-5.8%
  • 适合需要高安全性的视觉语言控制无人机系统

在快速发展的视觉-语言导航(VLN)领域,物理智能体的安全保障仍是开放挑战。为实现人机协同语言控制无人机的安全导航,系统需理解自然语言、感知环境并实时避障。控制屏障函数(CBFs)是保障安全运行条件的形式化方法,模型预测控制(MPC)则通过优化未来动作序列,在满足约束条件下实现平滑轨迹跟踪。本文针对视觉-语言操控的无人机平台,提出一种新型场景感知的CBF,利用带有红绿蓝和深度通道(RGB-D)的摄像头获取以我为中心的观测。基线系统采用视觉-语言编码器结合跨模态注意力,将指令转为地标序列,并通过目标检测验证生成路径。为增强安全性,本文提出自适应安全裕度算法(ASMA),可实时追踪移动物体,并在MPC框架内进行在线场景感知的CBF评估,作为额外约束。该系统持续识别潜在危险状态,实时预测不安全情况并主动调整控制动作,确保全程安全导航。在使用机器人操作系统(ROS)和Gazebo环境的Parrot Bebop2四旋翼上部署,相比无CBF基线,成功率达64%-67%提升,轨迹长度仅增加1.4%-5.8%。

原文摘要 · Abstract (English)

In the rapidly evolving field of vision-language navigation (VLN), ensuring safety for physical agents remains an open challenge. For a human-in-the-loop language-operated drone to navigate safely, it must understand natural language commands, perceive the environment, and simultaneously avoid hazards in real time. Control Barrier Functions (CBFs) are formal methods that enforce safe operating conditions. Model Predictive Control (MPC) is an optimization framework that plans a sequence of future actions over a prediction horizon, ensuring smooth trajectory tracking while obeying constraints. In this work, we consider a VLN-operated drone platform and enhance its safety by formulating a novel scene-aware CBF that leverages ego-centric observations from a camera which has both Red-Green-Blue as well as Depth (RGB-D) channels. A CBF-less baseline system uses a Vision-Language Encoder with cross-modal attention to convert commands into an ordered sequence of landmarks. An object detection model identifies and verifies these landmarks in the captured images to generate a planned path. To further enhance safety, an Adaptive Safety Margin Algorithm (ASMA) is proposed. ASMA tracks moving objects and performs scene-aware CBF evaluation on-the-fly, which serves as an additional constraint within the MPC framework. By continuously identifying potentially risky observations, the system performs prediction in real time about unsafe conditions and proactively adjusts its control actions to maintain safe navigation throughout the trajectory. Deployed on a Parrot Bebop2 quadrotor in the Gazebo environment using the Robot Operating System (ROS), ASMA achieves 64%-67% increase in success rates with only a slight increase (1.4%-5.8%) in trajectory lengths compared to the baseline CBF-less VLN.

无人机导航安全控制视觉语言强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。