用视觉语言模型指导无人机与地面机器人协同导航,提升复杂仓库环境下的自主避障能力。
SwarmVLM: VLM-Guided Impedance Control for Autonomous Navigation of Heterogeneous Robots in Dynamic Warehousing
- 通过视觉语言模型和检索增强生成动态调整阻抗控制参数。
- 在12次真实测试中实现92%的成功率,光照良好时目标识别准确率达8%。
- 无人机领航、地面机器人自适应避障,适合动态仓储场景的多机协同应用。
随着高效物流需求的增长,无人机(UAV)正越来越多地与自动导引车(AGV)协同工作。尽管无人机可在密集环境和不同高度间灵活移动,但其受限于电池寿命、载重能力和飞行时长,需地面机器人的配合支持。针对异构机器人导航问题,SwarmVLM通过阻抗控制实现无人机与地面机器人的语义协作。系统利用视觉语言模型(VLM)与检索增强生成(RAG)动态调节阻抗控制参数以应对环境变化。在此框架中,无人机作为领导者使用人工势场(APF)进行实时路径规划,地面机器人则通过可变拓扑结构的虚拟阻抗连接跟随,有效避开短障碍物。系统在12次真实场景测试中达成92%的成功率;在理想光照条件下,VLM-RAG框架实现了8%的目标检测与阻抗参数选择准确率。地面机器人优先避让短障碍物,偶尔导致横向偏离无人机路径最多50厘米,展现出在杂乱环境中安全导航的能力。
原文摘要 · Abstract (English)
With the growing demand for efficient logistics, unmanned aerial vehicles (UAVs) are increasingly being paired with automated guided vehicles (AGVs). While UAVs offer the ability to navigate through dense environments and varying altitudes, they are limited by battery life, payload capacity, and flight duration, necessitating coordinated ground support. Focusing on heterogeneous navigation, SwarmVLM addresses these limitations by enabling semantic collaboration between UAVs and ground robots through impedance control. The system leverages the Vision Language Model (VLM) and the Retrieval-Augmented Generation (RAG) to adjust impedance control parameters in response to environmental changes. In this framework, the UAV acts as a leader using Artificial Potential Field (APF) planning for real-time navigation, while the ground robot follows via virtual impedance links with adaptive link topology to avoid collisions with short obstacles. The system demonstrated a 92% success rate across 12 real-world trials. Under optimal lighting conditions, the VLM-RAG framework achieved 8% accuracy in object detection and selection of impedance parameters. The mobile robot prioritized short obstacle avoidance, occasionally resulting in a lateral deviation of up to 50 cm from the UAV path, which showcases safe navigation in a cluttered setting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。